Architecture for Validation

Validation can be performed at the level of a whole document, the current record, or a field within the current record.

Validation Architecture Diagram for Project35

Validation Scenario

1. The trigger for validation occurs when the user tries to save the current file, tries to export to final submission format, or tries to commit changes to the current record. All three of these triggers are initiated via the Form Generation System, which works to create deployments for the end applications. It does this by stepping thorugh an instance of a data model. When validation occurs, the application validates either the current record or the whole document.
2. During the validation process the application obtains descriptions via the Configuration Reader of various validation services. There are validation service applied based on the constraints described in the XML Schema. The remainder of validation services are derived from descriptions in the Configuration File, which have been associated with concepts in the XML Schema.
3. For each record and field, the Form Generation System queries the Configuration Reader within the Model Reader System to determine if there is a description of a validation service for the field or record being considered.
4. If there is a validation service description associated with the record or field, the Validation Services Factory provides this to the Form Generation System.
5. If there is no description of a validation service for the given, then the Form Generation System moves on to the next record or field.

Key Reference Sections

Form Generation System  |   Model Reader System  |   Configuration Reader  |   Validation Services Factory

Each section is structured to address purpose, description, design considerations, scope of effect, and relevant code packages.

Validation as a Specialised Service

There are three types of validation services that check the correctness of record data. Field level validation services validate the content of a particular field. For example, they could check that the value is a legal float number or that it matches a particular regular expression pattern. These services would be triggered whenever the user tried to commit changes to the current record.

Record level validation services detect incorrect combinations of field values within a given record. For example, suppose a form had fields for "gender" and "cancer_type". A record level validation service could be developed to detect an error for a "male" who had "cervical cancer". Like field level services, these services would also be triggered when the user tried to commit changes to the current record.

Document level validation services detect data entry errors in the entire document. These services could identify patterns in one record that don’t fit given the patterns of values found in another kind of record. For example, a document describing an experiment could have records describing a lab protocol that were inappropriate given the kind of sample described in another record.

Document level validation services would be triggered either when users explicitly wanted to view errors in the current document or when they attempted to export it to some final submission format. In the latter case, the presence of errors would prevent users from sending final draft documents to data repositories. This feature would help ensure that submissions had a high level of data quality.

Reference Sections for Validation

Form Generation System

Purpose	To generate UIs for Project35 which suite displays on desktop and Tablet PCs.
Description	Project35's architecture maintains a rigid separation between the structures that hold data and the structures that present them to an end-user. The model aspects are represented by the Native Data Structures, information for which can be found in the Core Architecture section. The view aspects are represented by the following packages: project35.desktopDeployment and project35.tabletDeployment. General Classes for Generating Desktop Project35 Forms When end-users invoke the "run_project35" script, it spawns an instance of Project35 Application. The object prompts end-users to load a model and it produces template record definitions that can be used to populate a document. Project35Application creates an instance of an empty Project35Dialog, which is the main window for the data entry tool; Project35Dialog has three major UI components: Project35MenuBar, NavigationTree, and RecordView; Project35MenuBar containing a variety of menus subclassed from Project35Menu; Project35Menu contains general-purpose code for handling plugins; NavigationTree is the tree display that appears on the left part of the window. This manages a tree of NavigationTreeNode objects that mirror the tree of RecordModel objects, which comprise the document; RecordView comprises a RecordViewTitle, which appears flushed left at the top of the panel, and a collection of DataFieldView objects, which render form fields; DataFieldView objects all use a corresponding DataFieldModel object that is part of the currently selected RecordModel object. General Class Relationship Diagram for Generating Desktop Project35 Forms Classes for Generating Edit Fields in Desktop Project35 Forms The image below describes a more detailed view of UI classes that are used to render text fields, identifier fields, combination box fields and radio fields. EditFieldView contains code which can render properties of a corresponding EditFieldModel object. It is responsible for rendering a "Plugins" button for any form field that has been associated with a collection of plugins. The other field view classes use properties defined in corresponding edit field model objects. RadioFieldView uses the choices provided by GroupFieldModel to render a group of radio buttons. If GroupFieldModel provides more than three choices, Project35 uses a CombinationFieldView to render the items as a drop-down list. URIFieldViews use TextFieldModel objects whose XML Schema definitions used "xs:anyType" for the type attribute (See EditFieldModel Properties). IDFieldView uses an instance of IDFieldModel to render a text field that is accompanied by a "Generate Key" button. The identifier service used to provide an identifier is a property of the IDFieldModel. Class Relationship Diagram for Rendering Edit Fields in Desktop Project35 Classes for Generating List Fields in Desktop Project35 Forms ListFieldView comprises a ListTypeManager, a ListValueManager and it uses an instance of ListFieldModel. ListTypeManager manages the type of child record that will be created or edited when end-users press "New" or "Edit" buttons. There are implementations of ListTypeManager that accommodate lists that support one or multiple types. MultiListTypeManager renders a combination box populated with the kinds of records that can appear in the list field. ListValueManager is an abstract class that manages the display of list items. SingleListValueManager represents a one item list as a single non-editable text field that shows the display name of the child record. MultiListValueManager shows the child records in a scrollable list of record names. project35.desktopDeployment.RecordViewFactory creates list fields. It determines the type of list field to produce by inspecting the fieldViewType attribute of the ListModel object. Class Relationship Diagram for Rendering List Fields in Desktop Project35 Classes for Generating Forms in Tablet Project35 Tablet Project35 was designed to support the same data models as those used to run Desktop Project35. However, we envisioned that different forms of deployment would be used at different stages of editing the same document. End-users will tend to use Tablet Project35 to do simple data entry tasks which require them to be at a work site such as a laboratory or a remote field location. They will tend to use Desktop Project35 to support complex data entry tasks and to fill in the rest of the document. Tablet Project35 was designed with the following principles in mind: Minimise the feature set of the application to support essential tasks; Minimise the use of pop-up dialogs; Economise on screen real estate; Change features which rely on right-menu clicks. The classes used to generate forms in Tablet Project35 are in the project35.tabletDeployment package. Many of them share the same name as other classes that cater to supporting Desktop Project35. There are a few notable differences. First, some of the menu items have been removed. For example, the File Menu does not contain an "Export to Final Submission Format" button. This feature was deemed non-essential because it was likely this would be done with the Desktop version. To reduce the number of pop-up dialogs, some features were altered so they used a stack of screens instead of separate windows. For example, consider the "Window" feature in Desktop Project35. When end-users switch from one file to another, a new window grabs focus. In Tablet Project35, only one file is shown at a time. When end-users change the current file, it changes the file loaded in the current window. As another example, the DefaultOntologyViewer was altered so it relied on JPanel objects instead of JDialog objects. This was done so the ontology viewer was more easily embedded in a stack of windows. To economise on screen real estate, the NavigationTree was removed. It is accessible via the "Where Am I" button, which pushes a view of the NavigationTree onto the stack of windows. Tablet Project35 supports navigation via a RecordStack object. RecordStack is a drop down list that shows the currently edited branch of the tree. Finally, the right click mechanism for activating ontology services is replaced by pressing a "Mark-up" button which appears at the end of a text field.
Design Considerations	Initially, Project35 tightly coupled data objects with the objects that viewed them. This led to severe performance problems because Java would potentially be managing thousands of UI components, each using a collection of Swing objects. This led to stripping the native data structures of references to UI components. Eventually the production of form field views was centralised in the class project35.desktopDeployment.RecordViewFactory. The most significant change in the form generation classes came when Tablet Project35 was developed. Initially we wanted to run the software on a PDA. However, these devices often require a specialised form of the JVM which does not support Swing components. Although the native data structures were stripped of references to UI components, they still had one crucial reference to the swing libraries: the ChangeListener class. We realised it would not be easy to port the code base to a PDA platform so instead we decided on a Tablet PC platform. It became clear that we had to make some changes to the forms. The NavigationTree was taking up too much room in the display and it was difficult to tap a pen on some of the nodes. Too many windows popped up and they cluttered the screen area. We also observed that it was difficult to activate ontology services. In the Tablet display, a right-click action is done by tapping and holding the pen on a form label. This seemed too awkward to do, so a "Mark up" button was developed instead. The development of TabletProject35 took 3 consultation meetings with mass spectrometer scientist Jennifer Lynch and ten business days of coding. The result proved that Project35's design isolated its model aspects enough to develop new ways of visualising them.
Scope of Effect	The UI classes are probably not used by anything other than classes in Desktop and Tablet deployment packages.
Relevant Code Packages	The classes used to generate user interfaces in Project35 are in the project35.desktopDeployment and project35.tabletDeployment packages.

Back to top

Model Reader System

Purpose	To coordinate the reading of the XML Schema and the Configuration File.
Description	This system is a construct for classification of coding areas that are found within the project35.mda package.
Design Considerations	We wanted a system that was capable of supporting a replacement of the Schema Reader. This can be accomplished by having the rest of the system interact with an abstract interface rather than a specific implementation something that interprets schemas. We needed a system that could extract information about data structures form the schema and another thing to extract information about form properties that were related to that schema. This system's role as a whole is to obtain enough information/properties to drive form generation.
Scope of Effect	Nothing else uses this as it's an abstract concept but parts of it will be used in other places as in the Configuration Reader. The components of the Model Reader System are used to synthesise Native Data Structures that are then used by the rest of the program. Most of the rest of the programme will only use the Configuration Reader.
Relevant Code Packages	project35.mda.schema, project35.mda.config, project35.mda.model (for more information on the last, see sections on Native Data Structures and Record Model Factory.

Back to top

Configuration Reader

Purpose	To manage options for configuring a data entry application that are not expressed in the XML schema.
Description	Project35 interprets the XML Schema to determine properties of the data entry application. However, many of the configuration options can’t be expressed in the XML Schema language, so they are managed in a ./[model]/config/ConfigurationFile.xml. This file maintains a collection of mappings that link schema concepts to different kinds of properties. Data modellers use the Project35 Configuration Tool to produce the file "ConfigurationFile.xml", which describes all the application properties that are associated with XML Schema concepts. When the Project35 application starts, it reads the configuration file and holds the information in instances of data container classes. These classes have properties which are analagous to classes defined in the XML Schema for the Project35 Configuration Tool (See the XML schema for the Configuration Tool in .\models\project35_form_configuration\project35_form_configuration.xsd of the distribution). Various parts of Project35 use the Project35 Configuration Reader to obtain configuration data that are used to render the application. For example, project35.desktopDeployment.TextFieldView uses the Project35 Configuration Reader to determine whether it should link an edit field defined in the domain schema to ontology services. In another case, Project35’s menu classes use the Project35 Configuration Reader to determine which standard menu items should be included for display. Other Configuration Files Project35 maintains two other configuration files in the config directory of a model folder: SessionAspects.xml and FileExtensionsToLaunch.xml The SessionAspects.xml file contains information about the most recently accessed files, and is managed by the class project35.mda.config.SessionManager. Project35 uses a list of recently accessed files to create the "Favourites" sub-menu located in the File Menu. It also uses session information to set the default starting directory for when users open files. The FileExtensionsToLaunch.xml file associates file extensions with shell commands used to launch other software applications. The mappings are used when users press the "View" button on a URL Field View. If the file specified in the text field ends with an recognised extension, Project35 will try to launch an application by making a system call. It remains the only configuration file that end-users still have to configure. Currently, the SessionManager uses the class FileLauncher to parse the file of mappings. In future, this file will be eliminated in favour of having the same information represented as properties in the Project35 Configuration Tool.
Design Considerations	In early incarnations of the software, data modellers had to craft the ConfigurationFile.xml file by hand. The file was awkward and time-consuming to maintain, especially when large XML schemas were being used. This led to the idea of using Project35 to edit its own configuration files. The Project35 Configuration Tool was developed using a schema of configuration properties and a collection of plugins which helped data modellers fill in the forms. The advent of the Project35 Configuration Tool meant that a configuration file could be designed far more rapidly than it could in previous releases. Enshrining configuration options in an XML Schema also meant it was easy to add new properties. The configuration options for Project35 expanded to include the selective inclusion of menu items and services which could be associated with a field, record or document scope of effect. With new features came data container classes that held the property values in memory.
Scope of Effect	The Configuration Reader is referenced in the file menu classes to determine what features should be included. Many of the classes which generate form fields interact with the Project35ConfigurationReader via the SchemaConceptManager interface.
Relevant Code Packages	Code for Project35ConfigurationReader and the data container classes can be found in the package project35.mda.config.

Back to top

Validation Services Factory

Purpose	To provide field level, record level, and document level validation services.
Description	Project35 supports validation services which can effect a field, record, or document. The Validation Services Factory is responsible for producing an instance of what is described here. Field and record validation services are triggered whenever the end-user attempts to commit changes to the current record. The exceptions are field level validation activities that check whether a required edit field has a value or if a required list field has one child record. In the Project35 tool, document-level validation services are only triggered when end-users try to export a data set to a final submission format or when they use the "Show Errors" feature in the View menu. Field level validation services are intended to identify problems in the value of an edit field or with the composition of child records found in a list field. Record-level validation services are intended to identify field values which are legitimate when considered in isolation but are wrong when considered in combination with other field values. For example, a form could have fields such as "cancert_type=ovarian" and "gender=male" which form an illegal combination of values. Document level validation services are intended to identify errors that appear in disparate parts of the same data set. The main validation utility used in Project35 is project35.soa.validation.ValidationFacility, which can be used to validate a field, record or document. The class has options for including or excluding certain types of errors from the validation activity. The project35.soa.validation package has a small hierarchy of interfaces shown below: Inheritance Hierarchy of Validation Service Classes The top level class is project35.soa.ServiceClass, which provides methods for setting and getting parameters. It also has a method for setting the resource directory, which is the default directory where files are expected to be found. Typically this will be the ./models/[project]/resources directory found in each model folder. The code base needs to be corrected so that all three of document, record and field level validation services subclass from ServiceClass. All the other validation service classes have methods that reference Project35FormContext. This is a HashMap that references a wide range of objects that are part of the Project35 application. For example, Project35FormContext has references to the RecordModelFactory, the NavigationTree and the currently displayed RecordModel. The HashMap allows services to use other parts of the application to inform how it proceeds with validation. DocumentValidationService has a method which expects the root record model in a data set. RecordModelValidationService has a similar method but it merely expects the current record model to be passed to it. FieldValidationService has fields for setting the field name and whether the field is required or not. The required field setting determines whether field validation services check that a field is empty or not. EditFieldValidationService is the same as ListFieldValidationService, except that the former expects a String field value and the latter expects a ListFieldModel object. Most of the classes in project35.soa.validation perform type-based error checks on fields. Subclasses of AbstractEditFieldValidationService check for double, integer and float type errors. Each of these services also has "bounded" versions which consider lower and upper limit values. The image below shows the variety of field level validation services that are automatically associated with the data type of a field specified in the XML schema and that provide basic type checking capabilities. Inheritance Hierarchy of Default Edit Field Validation Services The DateValidator class is special in that it relies on a static regular expression value for the date format. The value can be defined in the Project35 Configuration Tool. Once the value is set, all instances of the DateValidator check that a date value matches the specified format. The StringMaskValidator class is used to validate field values against a regular expression that is defined as a restriction in the XML schema. This provides a powerful way of validating form fields, such as constraining values to a certain number of characters or requiring values to have a particular prefix or naming convention. Many of the validation classes implement a ConstraintDescription interface. This human readable text is included in auto-generated functional specifications. In order to minimise the number of times validation services are instantiated, the ValidationServiceRegistrymanages all services created in the system. Service designers can use the Project35 Configuration Tool to specify whether a service should be persistent or transient. The ValidationServiceRegistry uses this configuration setting to determine whether it creates a new instance of a validation service each time it is asked, or if it returns the same instance of a service each time. Validation facilities in Project35 are extended by the Alerts System. To allow validation services and alerts to work together, the "validate" method for all validation services returns a collection of Alert objects which could represent errors or warnings.
Design Considerations	Originally, all the validation services dealt with errors in edit fields. As the project progressed, the interfaces for various services became more complicated. For Project35, all the service classes were overhauled so that services had a more consistent API. One of the most important changes was allowing service classes access to a wide number of application objects that might help guide their activities. More of this is covered in the discussion on Contexts in the Architecture Overview section.
Scope of Effect	The current schema reader associates most type checking services with record fields. Record and field validation services are activated in the following cases: The "Keep" or "Done" button is pressed on the main form; The "New" or "Edit" button is pressed in the list fields of the main form. project35.desktopDeployment.RecordView and project35.desktopDeployment.ListValueButtonPanel both show examples of enacting validation actions. Document level validation services are triggered when users try to use the "Export to Final Submission Format" button in the File menu or when they try to use the "Show Errors" button in the View menu. Code that calls validation services can be found in the FileMenu and ViewMenu classes found in both the project35.desktopDeployment and project35.tabletDeployment packages.
Relevant Code Packages	Most classes related to validation are defined in the project35.soa.validation package. Other classes appear in project35.soa.alerts. Classes that call validation services will appear in the FileMenu, ViewMenu, RecordView and ListValueButtonPanel classes of the project35.desktopDeployment package. The project35.tabletDeployment package contain similar classes.

Back to top