The ORM Foundation

I've been using ORM since the Visio for Enterprise Architect days, and have developed ad-hoc methods for using the tools to maintain a database over the course of its lifetime. I recall that I needed a six-page document to describe how to use VEA to maintain a SQL Server database some time ago. In that case, the documentation needed to describe ways to work around the various VEA  bugs.

That's no longer necessary, but I find I still lack a clear workflow. This is not as necessary when one or two developers are using the tools for their own use, but much more necessary when it's necessary to explain oneself to the new management.

I need something like this:

1. Reverse-engineer the database
2. Correct the NORMA bugs (like using "decimal" type for integer IDENTITY columns)
3. Rationalize the model with the cooperation of the domain experts
4. Create a new database from the result, add updated stored procedures, triggers, code, etc.

Then

1. Use NORMA to update the model for a maintenance release
2. Generate a new database?
3. Do what with the new database?
4. Migrate the changes to the Integration system how?
5. Maybe correct the model due to issues found during testing
6. Generate another new database
7. Migrate ...

Other related issues:

• Column order changes at random
• Constraint names change at random, and do not persist if changed manually in the Property Grid
• No way to specify things like defaults, "NOT FOR REPLICATION" and other things outside of the conceptual model
• etc.

I don't have a good answer for these issues. Perhaps others have also faced them and have processes of their own? I'd be glad to hear about them.

Thanks, John

John,

You raise a number of interesting and important points.
VEA has its strengths and weaknesses but it is "functionally stabilised" (as we used to say when I was with IBM.) so NORMA is the way we are going.
It is important to take on board that NORMA is a "CTP" and still has some way to go before it becomes a "product" but work continues.
If you think it would help, I'll consider putting a simple problem reporting subsystem on line for NORMA.  Let me know.

Anyway, from your post I interpret that your main challenge is to "explain something to new management" so since I have been doing this (on and off ) for quite a long time, I'll share my views on this aspect.

Step 1 is to get a very good understanding of how "management" perceive their problems.

If your management are "business oriented" then they will just want to know how to keep costs down and to deliver "results" in an acceptable timescale.
I have met many so-called "managers" who fit the characterisation of "Generals who know nothing about guns". So I think that it is a waste of time trying to explain anything about "methods" to such folks. Communication is more to do with (their) perception of and confidence in you than with any technical explanations.

On the other hand if "the new management" is more technically oriented and has some experience with application development practices, they may be looking for the perceived comfort of a "standards" based approach (i.e. UML) and will want to know why you are not using it. My research page gives some indications of potential answers to "the UML problem".  Again, this boils down to the simple fact that, despite the problems with VEA and NORMA, ORM is still much more effective than ER, UML and other over-hyped approaches.

So, if you want to continue this debate, may I suggest that you give us a bit more characterisation of the perceived needs of your "new management".

Ken

Ken,

First of all, I apologize: I should have made it clear that I haven't as much as set eyes on VEA in many years, probably not since 2003. I do not miss the necessity of carefully explaining how to avoid the bugs.

I've used NORMA for the past several years:I find that my first post in these forums was in March 2008. It just keeps getting better.

As I've only just met our new CIO this morning, I'll refrain from characterizing him beyond remarking that he walked around and greeted all of the employees of the IT department, asked us what we were working on, and seemed to understand what we were telling him. That's a very good start, in my book.

So, my question about explanation isn't as simple as "standards vs. not", but rather that I can't explain how I use the tool to modify the database over time. I find myself continually working towards a workflow that I could document, but not reaching one. At least, back in 2003, I was able to document my VEA workflow in only six pages, at least two of which pages were cautions on  how to avoid the bugs in the code.

A simple example is what I'm working on now: I added auditing columns to four tables (see "How to model log info?"), and have been struggling to do, with the tool, what I could have done by hand in a relatively short time. However, since I was developing the database model using the tool, I wanted to also use it to add these "infrastructure" columns. That's taking long enough that I might be required to explain why I'm taking so long.

Unfortunately, I'm having trouble explaining that to myself, much less to a manager (and hopefully not our new CIO!)

Hi John,

Thanks. Now I have a clearer idea of what you are looking for which I summarize as:
1: Defining a procedure for using ORM to maintain a database.
2: Being prepared to explain to your new CIO the reasons why you are using the ORM approach to maintain a database.

First "the DB maintenance procedure".
In our 2003 VEA book we described a VEA based DB maintenance procedure in chapter 16 and "How to do the people bits" in Chapter 17.
As you may recall, VEA had some tool support for what we called "three way synchronization".
If you take out the VEA bits, the procedure we described is the approach that I would still recommend today.
However, NORMA does not have support for this so you would have to do parts of it manually.
Since you are having trouble explaining the procedure to yourself, I recommend that you read Chapter 16 and then base your own procedure on this.
Then you should be able to see where the latest version of NORMA fits and does not fit.

Regarding the "people bits" (CIO included) - The approach we described in Chapter 17 is still valid today. (Except of course that you would use ORM2 diagrams instead of ORM1 and the few VEA specific bits are not relevant).

I see that Amazon.com has used versions of the book for $5 so this seems to be a cost-effective approach for you.

Other aspects of the "Be Prepared To Explain" challenge 
I see this in two parts: (a) Why ORM? and (b) Workarounds for NORMA 
(a) Why ORM: Terry has written several papers on this (some are in the Library) so you should be able to use these papers to complement your own knowledge and prepare a few PowerPoint slides. 
(b) Workarounds for NORMA: After following my earlier advice about the DB maintenance procedure, you should have an abstract process. 
It should then be "just" a matter of comparing NORMA's current functionality with the abstract procedure.

A "CIO" related story
In the late 1990's I was apponted as CIO of a multinational software development company which had several teams of developers in five countries spread across two continents. Some teams were DOS based C coders, others were using variants of "OO".
Only one of the teams (about 50 people) was using SQL Server so I focused my initial efforts on them.

Step 1: My first step was to interview every single person on the team. Amongst other things, I asked each of them what they were doing and had them show me some of their code and coding practices. However, not one of them reacted intelligently when I asked a "fuzzy question" about "normalization".

Step 2: Having gathered some information and made a start at establishing good 1:1 working relationships, my second step  was to figure out what I saw as "the problems to be solved"  and "solutions the problems".

Step 3: One of the things I did immediately was to create a new "database specialist" job and hire somone to fill it.
I had to interview amost 20 people before I found someone who could talk authoritatively about "all things database" including normalization.
Then I taught him how to use VisioModeler and in parallel I began to develop a "people friendly" approach to re-engineering the development practices of the whole team. (to be based on ORM of course)

This took a long time and in the middle of it all the Chief Architect resigned after I asked him to prepare a "CEO friendly non-technical"  presentation that explained what his brainchild software package was supposed to do indepedently of his usual "OO mumbo-jumbo-speak" explanation of how his brainchild did it. 

==================

From what you say, it seems to me that your new CIO has done "New CIO procedure step 1."
So, it is highly likely that your new CIO is now engaged in his own version of "Step 2"

In my case I had already been collaborating with Terry for many years so I knew the power of ORM and I was also a trusted member of the Board.
In your case, I have no idea of what your new CIO believes is "the correct approach" so I recommend that you find out as much about his background as you can.
For example: If your new CIO is enamoured of the IBM UML toolset then you may be in for a confrontation about why you are not using the "standard" Use Case approach. Same story for other "development methods"

Here are two relevant quotes:
Carl von Clausewitz: "Time spent in reconnaisance is seldom wasted"
Josh Billings: "It ain't what a man don't know that makes him a fool; it's the things he does know, that ain't so."

Hope this helps
Ken  

John Saunders:

I added auditing columns to four tables (see "How to model log info?"), and have been struggling to do, with the tool, what I could have done by hand in a relatively short time.

John, as has (I think) been noted elsewhere, you can create an AuditHistory entity type having a four-part identifier consisting of your four value types, and then associate an AuditHistory object to each audited object. Since the AuditHistory is "all key", it gets absorbed (or can be absorbed; you might need to set that manually) as four new columns in each of the relevant tables.