The ORM Foundation

I find it hard to know where to click on the line to enter data into the ORM Sample Popualtion Editor. If the current drop down editor is to be retained, then please cause clicking on the line to display borders around the dropdown, or even to open it.

Hi John,

I second that. That function is not user friendly the way it is. I think your suggestion is a good one.

I really like nORMa, but if there is a great student at Neumont who is interested in ergonomics/user friendliness, and a receptive staff member to give extraordinary credit for design over functionality, then please go for it. We need it!!! Everyone else, step back for just a bit.

I'm only guessing of course, but on the whole, I suspect it may be the case that there is little motivation to provide user friendliness as a 'top priority' within nORMa, unless user friendliness 'is' the thesis on which the student/programmer is paying attention to (rather than getting great kudos for developing DML feeds etc, which is totally understandable from my point of view. Who doesn't want kudos?).

It's a great product, and we only need to look at Job's billions to see that design has tremendous value today and should 'be' where the kudos is.

Anyway, that's my bit. Thanks for speaking up John.

best regds
Victor 

No disagreement that we need to fix this. There is a lot going on in this dialog beyond just typing data in cells. First, population values in NORMA always resolve back to an underlying ValueType at some level, so there are existing values to choose from and match. Secondly, you get the issue of complex identifiers (nested complex identifiers, etc), which resolve to multiple underlying values, resulting in complex item expansions that enable both pick lists and inline creation. At the surface of just typing in a few values it looks like a trivial problem, but things rapidly get more involved in the complex cases. Here are a few things that I don't like, followed by some of the technical issues that need to be resolved to fix them.

1. I'd like to see the dropdown active when a cell is activated, regardless of whether or not a textbox is supported. This gives an indication that there is data available in the dropdown.
2. There is little reason to have an edit field active for readonly data, the dropdown will do. Having a readonly edit field active is misleading.
3. Typing in a cell should move it directly to edit mode, if available.

There are a handful of reasons I haven't fixed this up yet. If the solution were trivial I would have fixed it long ago. This will fall on the SPD list after the ability to do instances of subtypes, which is also a non-trivial workitem (I have that one well on its way, but it wasn't ready for the May 2008 drop).

Technical issues. (You may not care, just showing that I'm not slacking):

1. There are four levels of activation in VirtualTreeGrid control that is used for the sample population editor. Note that any of these go well beyond what a standard tree control would offer (Explicit and Delayed, in place basic textboxes only). Explicit (triggered by code or the standard F2 key), Delayed (click with the mouse, it wakes up eventually), Immediate Mouse (same as delayed without the delay), Immediate Selection (activation occurs when a cell is selected by any means). ImmediateSelection is basically what you're asking for, and something we do in other windows (the New... dropdown in the Reading Editor is ImmediateSelection, for example). The issue is that any immediate selection inline controls need to extremely careful to not interfere with normal tree and/or grid navigation, so the mode is not generally used with controls that activate TextBoxes, which eagerly eat navigation keystrokes.
2. There is a nasty WinForms bug that causes problems with focus when an inline control is deactivated because of a focus change. Basically, the framework doesn't check that you're already losing focus and sends focus somewhere else. If you're wondering why you hit Escape and the toolwindow deactivates, this is why. These issues need to be handled on a case-by-case basis, and even then it can be pretty touching. I reactivated this bug repeatedly inside Microsoft, but I left before I could bully it through internally and they've never fixed it.

To fix these items and get the behavior we all want, I need to extend the existing control (or create a new one) that is active and waiting for keystrokes, but does not eat navigation keys or show the edit window until something other than a navigation key is typed, the control is clicked, or F2 is pressed (typical spreadsheet behavior, but this an instance pick-list dropdown on each). I obviously need to figure out the focus issue as well so that escaping an edit doesn't send you focus to Timbuktu.

In the meantime, the keystrokes to know are F2 (activate an edit box) and Alt-Down (if you have a dropdown button, then open it).

Hopefully that gives you a better feel for where I'm trying to go. I'll let you know when I'm there, but the comments on this issue make it clear that I need to do something soon to make sample population workable.

-Matt

I'll be the first to admit that there are usability improvements to be made in the tool, but also realize that the core functionality is extremely important, and we have very agressive goals, such as live incremental relational models on top of an ORM model (not the snapshots of yesteryear). We have quite a few design and usability improvements we want to do moving forward, but functionality is trumping at the moment. I'm hearing a lot more requests for complex constraint DDL, finished verbalization, and customizable column order than for improved autolayout. It's just a matter of prioritization, and an architecture where the view(s) are completely separated from the underlying models (yes, we can actually load an validate an ORM model without Visual Studio running). We'll get a pretty Relational Designer when the generated columns/tables are incremental, but not before.

I'd much rather have towing capacity than a chrome bumper in front of an old four-banger. I can add the chrome later, but it's a lot harder to stretch the chassis and replace the engine.

It's a work in progress. Design has not been ignored. Also realize that a lot of what you're objecting to just isn't done yet, often due to major framework extensions that need to be made. Obviously, the feedback here helps establish prioritization, and I'd like to hear any targeted usability comments and suggestions on other areas.

-Matt

Hi Matt,

You did a good job of explaining the factors involved in the development of a better sample population editor.  I think most knew that the priorities for the nORMa project, as a whole, were going to be (and ought to be), in the core, rather than UI elements.  The new information was about the technical complications to including the functionality improvements we probably all want.  The limitations of the .Net framework are there, and an app built on any framework has to face such constraints.  You also put forward the challenges of complicated validation issues inherent with ORM populations.  Typical validation is a pretty straight forward comparison of values or format (against some established standard - or at worst a changing value that is easy to determine).  With the nORMa tool, the validation is against a dynamic model with all the rules of base theory to take into account.

I'd stick with the emphasis on the core "engine" and the principle of the "low lying fruit" to improve the UI elements when the opportunity presents itself.  In that vein, a few of my thoughts on sample population entry:

• An application error in sample population validation makes the feature worse than useless (in that it gives a false sense of correctness). It's like having a handy calculator, or tax software that sometimes misplaces a decimal point. There are no trivial errors in a relational model schema. The logical correctness provided by the ORM methodology is the reason we follow and appreciate the development of the tool.
• There may be two ways of considering sample population validation.
• Your post mentions a number of issues with validating entries against the model - with the problems of the moving target as the model evolves.
• There can also be validation against the existent values in the domain/UofD. The real concerns for validating against the model are not inherent in validating against values already assumed valid in the domain. These can be taken as already vetted in both form and content. It may make it easier to get valid sample population values into the application, if this second path is incorporated. Yes, errors in the domain values will propagate through the model; but that's no worse than propagating a misstated rule of the UofD. In both cases, that responsibility is outside of the tool. --- I don't favor reverse engineering of data structures to get a conceptual model (I think the rationale is flawed); but as a way to harvest domain valid sample population values, it could be useful.
• A problem I have with the current system is that it's non-selective. If you use sample Employee ID numbers to validate one Fact Type, you get flooded with errors from every other Fact Type using the Employee(.id) Object Type (where you have not entered any sample population value). This is technically correct, but annoying. I don't know if the validation can be made selective on a Fact Type basis, or if that would be trivial (somehow that doesn't seem likely); but keep the idea in mind for later. It may even tie in with the domain validation mentioned above.

As with any project under development, it's helpful to those using the previews to know which parts are known correct, and which parts are included, but should not be trusted to the same degree.  Other than some color coding scheme, I guess you're going to have to keep us informed via these forums - thanks for the perspective,

BRN..

Brian Nalewajek:

There can also be validation against the existent values in the domain/UofD. The real concerns for validating against the model are not inherent in validating against values already assumed valid in the domain. These can be taken as already vetted in both form and content. It may make it easier to get valid sample population values into the application, if this second path is incorporated.

Certainly an idea worth exploring. I'm most interested in prepopulating a sufficient (TBD what that means) number of samples from an existing DB to validate any constraint we can reverse engineer. Of course, the concern here is that you end up relying on existing data to build and validate your model when you really need to pull back and carefully consider the individual FactTypes and constraints. I don't want to go down the path of normalization theory, which is basically useless unless you have a fully representative population. However you're supposed to either get sufficient samples on a big model and prove that the population is sufficient without a structure to store it in is something best left for the normalization theorist. I'd rather use a carefully validated ORM model to bypass the whole field.

Brian Nalewajek:

A problem I have with the current system is that it's non-selective. If you use sample Employee ID numbers to validate one Fact Type, you get flooded with errors from every other Fact Type using the Employee(.id) Object Type (where you have not entered any sample population value). This is technically correct, but annoying.

This is discussed extensively in my last post on another sample pop thread. Just turn off the 'Sample Population Mandatory' error display and you can share values, but otherwise isolate The FactTypes from each other.

-Matt

Hi Matt,

Thank you for getting back to John and I on this one. Of course I support John's thread to tickle for debate, but I am a user of nORMa as much as authoring (what I hope to be) some friendly rivalry with Richmond. I have read the licence agreement for nORMa, and I think it's a good thing that in order to (ostensibly) get a working 'standard' in the market, it can be done in such a way that benefits the rest of the community. To date, we haven't used any of the code, or even looked at it for that matter, but I'm having a good look at the meta-model under the precepts of the call for a 'common meta-model'. I am not sure how we do that (achieve a common meta-model) as yet, but I'll work with the community on that one. And to be honest, and overall, I haven't got a problem with how it looks at the moment. There are only one or two little things that I'll bring up for discussion as soon as I can find time.

Brian - My apologies, I beg to differ on this one. In that, in the absense of a voted spokesperson for 'all' or 'most' of the ORM community, I can only imagine that every man/woman speaks for themselves. i.e. I agree that it's worth looking into the usability. And again, 'thanks Matt', it'd be good if someone can have a look at that one small thing at some time. From our perspective, and on usability in general....certainly Viev wouldn't have started Richmond if nORMa did everything as quickly and as easily as we wanted. Personally, and only personally, I care very little about any other factor than usability (at any level). i.e. We're not competitive just for the sake of competition.

I agree that it's a shame about the restrictions within the .Net framework. I read about those in the old SF forum for nORMa. But, who knows, that may change, or someone may think of a really smart way of getting around things. VS changes all the time.

Anyway, We have got to the stage at Viev where we are now writing a 'reading editor' and realise the mountain you must have climbed to get there. So well done on what it is today. If it can be improved, as per John's start to this thread, why not? All the better I say.

Best regds

Victor

I wouldn't read what I've said as a slam on .NET/DSLTools at all. It doesn't mean I can't do what I've mentioned, just that it doesn't come free out of the box. We've written quite a few framework extensions ranging from delayed validation rules engines to multi-diagram support to a custom Model Browser and extensible serialization engine. However, 'out of the box' pieces were very good, but we're obviously going well beyond most simple DSL requirements and essentially building our own platform on top of it. A number of pieces are very good. For example, you mention editor capabilities. The 'Fact Editor' is using the same framework as several of the shipped MS editors, including all of the XML editors. We've barely scratched the surface on what it can do.

The main reasons for choosing VS/DSL were personal background (Terry worked on the core metamodel for the framework, and I obviously had a lot of traction with the SDK), a decided VS-preference among our students, and some lacking pieces in EMF that I couldn't divine from the spec. Specifically, the live incremental validation work we're doing requires delete closures and the associated deleting notifications, which need to be implemented at a very low level in the framework. I'd like to be wrong on this (please point me to some documentation if I am).

To be perfectly honest, the metamodel is just the start. The metarules are where the real action is at and are the areas that require the majority of code. We've considered walking away from DSL and writing our own modeling/rules engine that would run across multiple platforms and languages, including cases where no traditional development environment is loaded at all. Given that the majority of rules rely heavily on the generated model and barely touch .NET (beyond things like List and Dictionary), and our ability to generate code, transform code, and rerender it in different forms (including additional languages), we're technically well withing range of writing our rules in .NET and automatically respitting them in Java--but only if we have a shared model with sufficient capabilities.

Happy to hear metamodel comments. We certainly have a few of our own, so you might want to start with short queries asking for the justification/future of certain elements. You might find that it isn't worth your time tearing apart something we already know is flawed.

-Matt

Wow, I really seem to have stirred up a firestorm with what I thought was an innocent comment. Good discussion, though.

I'll be re-reading the comments about the DSL tools, as I've been evangelizing them here at work (with some success). I'm also customizing the Web Service Software Factory, which has implemented some interesting techniques on top of the DSL Tools and the Guidance Automation Toolkit. If nORMa people aren't already doing so, then "your people should talk to their people". You should find a lot to talk about.

As to functionality vs. usability, I lost track of the project status for 1/2 year or so, and didn't have a clear idea of what was meant to be working and what was not. I'll try to stay closer in touch, so I don't report usability issues with incomplete software. I've had experience in "serving two masters" with having to support people using software of beta quality or less in production environments, at the same time as you're trying to bring the beta software up to production quality. It doesn't work very efficiently.

John Saunders

The DSL and factory approach (reduce the problem to a datamodel, represent in a structured (often graphical) fashion, and then generate) is definitely the wave of the future. The main complaint on DSL is that it doesn't go as far as we need. For example, we need multiple diagrams, multiple shapes per diagram, a richer rules engine, etc. The main issue, however, is the assumption at several levels of DSL (such as the provided serialization engine), that you know the full set of domain models at compile time (we don't) and that there is a single clean mapping from an ObjectModel to a ShapeModel (for example, you can only draw an ElementLink with a Connector, which makes things like drawing a SubtypeFact a hack). DSL is great if you know your full set of data up front and the model size is easily manageable--neither of which are a fit for NORMA without framework extensions.

If you're interested in some the DSL rule extensions we've done I'd be happy to share a couple of white papers, but I don't want to turn this forum into a DSL discussion site. You might also be interested in things like the debug transaction viewer dialog, which are easily plugged into any DSL project. I would also like to take more time looking at the GAT as a way for smoother NORMA integration into existing and new projects. Any pointers on the areas that have been most useful to you.

We try not to add pieces to setup that are not at least usable and stable. 'Complete' and 'polished' are not always possible, but stable and usable should be. I hadn't realized the Sample Population Editor was essentially too weak for use until late 07, when I spent some time stabilizing it, but did not add significant functionality (changeset 1173 if you want to check comments on Sourceforge). Clearly, there's interest in a little split and elbow grease, so you should see things improving significantly, hopefully satisfying most of the concerns raised in the various recent threads.

-Matt

 Hi John,

Just for my own personal use of a tool, I wanted an 'Intellisense' like interface for 'Predicate Reading' population.

Anyway, it was quite challenging programming, but we've done our level best to provide a new vision on 'Predicate Reading' creation.
At the end of the day I was surprirsed though, as it seems (having done it now), that 'Predicate Reader' forms will look more or less the same from tool to tool. Speed of creation may differ though, based on the technology used.

Here's a link to a new post (here at OF) that may be of interest. 

        http://www.ormfoundation.org/forums/p/398/1124.aspx#1124

Best regds

Victor 

 Hi Victor,

Sounds good; I'll have to take a look.  Matt mentioned a year or so back, how difficult adding something like Intellisense can be.  I also like that you are giving predicates some attention.  I think I mentioned that LINQ queries have the "FROM clause" at top, just to enable the use of Intellisense.  Some form of that technology has to be part of a practical ORM tool.  Sample Population entry looks like a prime candidate for the ease of use improvements that this should make possible.

BRN..