Overview Sections:  Overview of Architecture*  |   Service Anatomy  |   Service Families  |   Accessing Application Variables

Detailed Architectures:  Core*  |   Validation  |   ID Generation  |   Ontologies  |   Meta Data  |   Functional Specs  |   Alerts  |   Plugins
* If you are looking at this documentation for the first time, we recommend you start here for each section.


Architecture for the Core

The Core of Project35 is the minimum needed to run the application. With this, you can generate forms for immediate use either for data entry or for requirements gathering. To extend the functionality and capability of the core, read through the other sections of the architecture and the rest of the design manual.


Core Architecture Diagram for Project35


Core Scenario

  1. The Model Reader System tries to determine properties in the generated application. This information is spread out betwen the XML Schema and the Configuration File.
  2. These files are read by their respective readers to create templates of records that will hold form data (native data structures).
  3. These native data structures are managed by the Record Model Factory (the main management function is creation). These templates are used by various parts of the application.
  4. The I/O System uses the data structures to populate a data set. Native data structures are also used when users create new records using the forms. The population of these data structures (to form data sets) can be done by the reading components that are managed by the I/O System. This I/O System uses the native data structures to populate a data set whenever it uses the native format of final submission form at reader.
  5. These populated data sets are then visualised by the Form Generation System. The general structure of the data set is represented by a tree. This is the record tree. The currently selected record is represented by a form.
  6. The form is rendered according to two factors: whether it is rendering for a tablet or desktop, and by querying to the Configuration Reader for information about each field.


Key Reference Sections

Model Reader System  |   Schema Reader  |   Configuration Reader  |   Configuration Tool  |  
Native Data Structures  |   Record Model Factory  |   I/O System  |   Form Generation System

Each section is structured to address purpose, description, design considerations, scope of effect, and relevant code packages.


Inputs and Outputs of the Core

Inputs to Project35 come from an XML Schema and a Configuration File. The XML Schema can be produced with any editor you wish to use. The Configuration File is the product of the Project35 Configuration Tool. Both of these inputs provide information for the generation of data entry forms.

The outputs of Project35 are forms that are automatically generated from the model provided at the beginning. These forms are accessed by means of one of two deployment applications: a Desktop and a Tablet.


Reference Sections for the Core

Model Reader System

Purpose To coordinate the reading of the XML Schema and the Configuration File.


Description This system is a construct for classification of coding areas that are found within the project35.mda package.


Design Considerations We wanted a system that was capable of supporting a replacement of the Schema Reader. This can be accomplished by having the rest of the system interact with an abstract interface rather than a specific implementation something that interprets schemas. We needed a system that could extract information about data structures from the schema and another thing to extract information about form properties that were related to that schema. This system's role as a whole is to obtain enough information/properties to drive form generation.


Scope of Effect Nothing else uses this as it's an abstract concept but parts of it will be used in other places as in the case of the Configuration Reader. The components of the Model Reader System are used to synthesise Native Data Structures that are then used by the rest of the program. Most of the rest of the programme will only use the Configuration Reader.


Relevant Code Packages project35.mda.schema, project35.mda.config, project35.mda.model (for more information on the last, see sections on Native Data Structures and Record Model Factory.

Back to top


Schema Reader

Purpose

The schema reader sub-system combines information from the XML shema and the Configuration File to create templates of native data structures. These structures are instantiated whenever a new record is needed by the generated data entry tool.



Description

The application relies on an implementation of a SchemaReaderInterface, which is responsible for transforming XML schema structures into native data structures that the rest of the program uses. Currently the SchemaReaderInterface is implemented using Sun's MSV Schema Reader, although Project35 is designed to support other schema readers. Historically the application code has had to change schema reader technologies multiple times, so the application was re-designed to decouple the technology for reading schemas from the technology used to manage record data in the rest of the application.

The XML Schema is used to express properties such as the data types of edit fields, the cardinality of list fields and whether fields are required or optional. There are many other application properties supported by the application which cannot be expressed in the XML schema language. The architecture maps records and fields defined in the schema with properties defined in a configuration file. This configuration file is edited via the Project35 Configuration Tool. The properties are interpretted via the Project35ConfigurationReader class.

The schema reader is responsible for combining the information provided by the schema and the properties provided by the configuration reader to create template definitions of data structures that hold data. These native data structures are treated as factories, and are used whenever the users load or create new records for their data sets.

The schema reader is used once to interpret the XML schema and the configuration file created for each data entry model. After it finishes its activity, the rest of the application is driven off the record factories. The native data structures are used for two purposes:

  • To generate the data entry forms (see the Form Generation System section for more);
  • To read and write data to files(see the I/O System section for more).



Design Considerations

Historically, the template record structures contained many properties which were not necessary to be known by every instance. For example, designers may associate the help file "PhoneNumber.html" with the "phone_number" field of the record "contact". It is not necessary for each instance of the contact record to know about the HTML help file.

The unecessary references to properties took up memory space and caused problems when the data objects were serialised during copy/paste operations. Unfortunately, if you're serialising an object, all of the objects it uses must also be serialisable. In early versions of the code data fields held a reference to instances of services defined in the configuration file. When the field was serialised, the application generated errors because implementations of the services were not serialised. To avoid this problem, the data structures were stripped of many properties that were later managed through a lookup feature in Project35 Configuration Reader.

In activities such as form generation, the application now uses the record type and field name of the current form object to look-up the HTML file that should be made available to end-users.



Scope of Effect

The Schema Reader system is only used once when an application starts. Almost all the application code relies on the templates of record definitions managed by project35.mda.model.RecordModelFactory and does not interact with the schema reader.

Replacing the schema reader would require rigorous unit testing because of the potential for errors when the schema reader uses model properties to set attributes in the template record definitions. The activity lends itself to automated testing and will not require testing of the application itself. For example, test cases could run the schema reader and then perform JUnit test cases that verify that records have the expected number of type of fields.



Relevant Code Packages

There are three relevant packages that support the activity of reading the XML schema and its associated ConfigurationFile.xml:

  • project35.mda.model
  • project35.mda.schema
  • project35.mda.config

The class project35.mda.schema.Startup is used to encapsulate much of the activities that involve:

  • Having the user select a model folder;
  • Interpretting the XML Schema;
  • Interpretting the Configuration File.

Startup provides the SchemaReaderInterface with the name of the XML Schema file typically found in ".\models\[your model]\model" and the configuration file found in ".\models\[your model]\config\ConfigurationFile.xml"

Currently, Project35 uses project35.mda.schema.MsvSchemaReader as the implementation of the SchemaReaderInterface in the same package. The class uses Sun's MSV schema reader to parse XML schemas. Schema properties are used to create definitions of native data stuctures that include classes like RecordModel, ListFieldModel, EditFieldModel. More information can be found in the Native Data Structures section.

Once MSVSchemaReader has finished defining basic data structures, it adds properties that are provided by project35.mda.config.Project35ConfigurationReader, the class which reads ConfigurationFile.xml.

Project35ConfigurationReader parses the configuration file by delegating to helper classes such as:

  • ServiceConfigurationParser
  • MenuFeatureconfigurationParser
  • RecordConfigurationParser

Configuration file options are stored in classes whose names end in "Configuration", such as ListFieldConfiguration, EditFieldConfiguration and RecordConfiguration.

Back to top


Configuration Reader

Purpose To manage options for configuring a data entry application that are not expressed in the XML schema.


Description

Project35 interprets the XML Schema to determine properties of the data entry application. However, many of the configuration options can’t be expressed in the XML Schema language, so they are managed in a ./[model]/config/ConfigurationFile.xml. This file maintains a collection of mappings that link schema concepts to different kinds of properties.

Data modellers use the Project35 Configuration Tool to produce the file "ConfigurationFile.xml", which describes all the application properties that are associated with XML Schema concepts. When the Project35 application starts, it reads the configuration file and holds the information in instances of data container classes. These classes have properties which are analagous to classes defined in the XML Schema for the Project35 Configuration Tool (See the XML schema for the Configuration tool in .\models\project35_form_configuration\project35_form_configuration.xsd of the distribution).

Various parts of Project35 use the Project35 Configuration Reader to obtain configuration data that are used to render the application. For example, project35.desktopDeployment.TextFieldView uses the Project35 Configuration Reader to determine whether it should link an edit field defined in the domain schema to ontology services. In another case, Project35’s menu classes use the Project35 Configuration Reader to determine which standard menu items should be included for display.

Other Configuration Files

Project35 maintains two other configuration files in the config directory of a model folder: SessionAspects.xml and FileExtensionsToLaunch.xml

The SessionAspects.xml file contains information about the most recently accessed files, and is managed by the class project35.mda.config.SessionManager. Project35 uses a list of recently accessed files to create the "Favourites" sub-menu located in the File Menu. It also uses session information to set the default starting directory for when users open files.

The FileExtensionsToLaunch.xml file associates file extensions with shell commands used to launch other software applications. The mappings are used when users press the "View" button on a URL Field View. If the file specified in the text field ends with an recognised extension, Project35 will try to launch an application by making a system call.

It remains the only configuration file that end-users still have to configure. Currently, the SessionManager uses the class FileLauncher to parse the file of mappings. In future, this file will be eliminated in favour of having the same information represented as properties in the Project35 Configuration Tool.



Design Considerations

In early incarnations of the software, data modellers had to craft the ConfigurationFile.xml file by hand. The file was awkward and time-consuming to maintain, especially when large XML schemas were being used. This led to the idea of using Project35 to edit its own configuration files. The Project35 Configuration Tool was developed using a schema of configuration properties and a collection of plugins which helped data modellers fill in the forms.

The advent of the Project35 Configuration Tool meant that a configuration file could be designed far more rapidly than it could in previous releases. Enshrining configuration options in an XML Schema also meant it was easy to add new properties.

The configuration options for Project35 expanded to include the selective inclusion of menu items and services which could be associated with a field, record or document scope of effect. With new features came data container classes that held the property values in memory.



Scope of Effect

The Configuration Reader is referenced in the file menu classes to determine what features should be included. Many of the classes which generate form fields interact with the Project35ConfigurationReader via the SchemaConceptManager interface.



Relevant Code Packages Code for Project35ConfigurationReader and the data container classes can be found in the package project35.mda.config.

Back to top


Configuration Tool

Purpose The Project35 Configuration Tool provides application designers a graphical application for setting the values of configuration options in generated data entry applications.


Description

Project35 generates data entry tools based on the record and field concepts specified in an XML Schema. The XML schema language is able to describe properties of form fields such as:

  • The basic data type;
  • Whether the fields should be rendered as an edit field, an attribute field or a list field;
  • Whether a field is required or optional.

However, the schema is not expressive enough to describe all of the application properties that are supported by a data entry application. For example, there is no facility in XML schema which could be used to associate a context help file with a given record or field.

Features that cannot be inferred from the XML schema are managed in a configuration file which has a path of .\models\[your model]\config\ConfigurationFile.xml. Project35 maps record and field concepts described in the XML schema to application features described in the configuration file.

ConfigurationFile.xml stores configuration data for features including:

  • The title of the data entry application;
  • Colours for aspects of the UI, including colours for buttons, panels and text;
  • The kinds of data sets the application can import;
  • Some standard menu features which can be turned off and on.

The ConfigurationFile.xml file also manages the links between parts of the generated applications and services which include Ontology Services, Validation Services, General Purpose Plugins, and ID Generator Services.

The properties of ConfigurationFile.xml are defined in an XML Schema. Project35 uses this schema and its own form generation engine to generate the Project35 Configuration Tool.

Architecture of the Project35 Confguration Tool

The above image presents a simplified version of the system described in the main Architecture section of this manual. The software uses the following two files to generate the Project35 Configuration Tool:

  • XML Schema - found at .\models\project35_form_configuration\model\project35_form_configuration.xsd
  • Configuration File - found at .\models\project35_form_configuration\config\ConfigurationFile.xml

The above image shows the various kinds of services which were developed to provide configuration features. Developers should find good examples of how to make services by examining the class files that were developed to support these features.


General Purpose Plugins

The following services implement the project35.soa.plugins.Project35Plugin interface.

Class Description
DefaultConfigurationFilePlugin Document-level plugin. Automatically creates a stubbed form of a configuration file. The plugin examines all the fields in all the record templates managed by the RecordModelFactory object. It then creates blank configuration records for each record and field entry it finds.
ContextHelpPageCreationPlugin Document-level plugin. Fills in the "help_link" field of the configurations for records and fields. It provides application designers the option of specifying locally or remotely maintained help files. If the creation plugin is set to use locally used pages, it attempts to generate skeleton help file documents in .\[your model]\doc directory.
DefaultOntologyServiceDescriptionsPlugin Field-level plugin. Used to create configurations for default types of ontology services such as those which use tab-delimited files. The feature is meant to help reduce data entry errors introduced by manually creating the ontology service descriptions.
GenerateFunctionalSpecificationsPlugin Document-level plugin. Used to generate a human-readable functional specification based on the XML schema and the currently edited configuration file. See Functional Specifications System for more information.
OntologyIdentifierCreationPlugin Document-level plugin. Used to associate record and field entries in the XML schema with ontology identifiers. Ontology services can be designed so they use the identifiers to help scope the terms they return to end-users.
TestApplicationPlugin Document-level plugin. Used to generate a test application based on the current state of the currently edited configuration file.


Ontology Services

The Configuration Tool relies on an ontology service which uses Project35ConfigurationOntologySource. The service allows service designers to populate the "name" fields for record and field configurations.

Validation Services

The Configuration Tool uses two document-level validation services:
  • DefaultFieldConfigurationsValidationService - ensures that for any record configuration, there are no duplicate field configuration. Duplicates may arise from manual data entry errors.
  • DefaultRecordConfigurationsValidationService - ensures that the configuration file does not have duplicate record configurations.

Originally this ConfigurationFile.xml was edited by hand. However, the options in the file can be defined in an XML Schema. The software was modified to use its own form generation engine to create a graphical Configuration tool for application designers.



Design Considerations

Initially, the configuration files were coded by hand. Over many years, it became clear that manually producing the ConfigurationFile.xml was an obstacle to widespread use of the software.

Using Project35 to produce a Configuration tool for Project35 provided a number of benefits. First, it helped reduce the amount of effort spent coding, testing and documenting a new utility tool. Second, using the software to build its own Configuration tool provided an opportunity to test two things:

  • The ability for the architecture to support enough customisations to produce a tool that could be used for a production-line activity;
  • The core code base. Many bugs were discovered during the development of the Configuration Tool.

Finally, the design supports future extensions of the Configuration Tool. See Architecture Extensions to understand how easy it is to make Project35 support more configuration options.



Scope of Effect

Adding, changing, or deleting the plugins will impact two aspects of development:

  • Code that is part of the project35.configurationTool package;
  • The configuration file for the Configuration Tool: .\models\project35_form_configuration\config\ConfigurationFile.xml.



Relevant Code Packages

All of the code for the Configuration Tool relates to the plugins it supports. The code for plugin features resides in the project35.configurationTool package.

Back to top


Native Data Structures

Purpose The native data structures are an in-memory representation of an XML data set.


Description

The basic native data structure in Project35 is the RecordModel class, which comprises a number of DataFieldModel objects. The image below shows the different kinds of data fields that can be created:

Inheritance Hierarchy for Data Field Classes Supported in Project35

All fields will have properties such as:

  • A field name;
  • A help link for a URL;
  • Whether the field is required or optional;
  • The kind of field view type (eg: "RADIO_FIELD", "DATE_FIELD" etc);
  • Whether the field is an attribute or not (this probably belongs in EditFieldModel class);
  • Text to appear when an end-user hovers over the field label.

The two kinds of fields are EditFieldModel which manage a single value and ListFieldModel which contain one or more RecordModel objects. EditFieldModels will have:

  • A value represented as a string;
  • A default value that should appear when forms are rendered with a new record model object;
  • Whether the field value should be included as part of the display name that represents the containing record.

ListFieldModel objects will know what kinds of record models they can contain and will have a collection of children RecordModel objects.

There are four kinds of EditFieldModel subclasses, although they have few extra properties. A GroupFieldModel is a kind of EditFieldModel where end-users select a value from a list. Its subclass BooleanFieldModel constrains this list of choices for "true" and "false". TextFieldModel is a marker class for identifying fields that can be associated with ontology services or id generation services. An IDFieldModel is a kind of TextFieldModel which also has an IDGeneratorService. This service creates an identifier value that can be inserted into the form field. An IDFieldModel is the data container object that corresponds to an attribute field in XML Schema.

A RecordModel is a collection of DataFieldModel objects. The image below shows how the RecordModelFactory, RecordModel, EditFieldModel and ListFieldModel classes relate:


Aggregation Relationships for Project35 Native Data Structures

A RecordModel will contain a collection of EditFieldModel objects and a collection of ListFieldModel objects. Each ListFieldModel can contain multiple RecordModels. When the SchemaReader is operating, it adds RecordModel instances to the RecordModelFactory. The instances act as templates that can be cloned whenever a new record model is needed for a data set.

The most popular utility class in the project35.mda.model package is RecordModelUtility. It has a number of methods to help group fields in different ways.

There is a strong relationship between properties of the XML schema and properties of the native data structures. The following tables list the attributes of native data structures that should be set with values derived from a schema:


RecordModel Properties

Property record_class_name
Description The name of the record.
Provider
  • <xs:element name="[record_class_name]"> 
    <xs:complexType>
    	<xs:sequence>...
    </xs:sequence>
    </xs:element>
    


Property helpLink
Description The URL for a web page that describes the form concept.
Provider


Property ontology_identifier
Description A unique identifier that can be associated with a schema concept. This is useful if ontology services want to use a schema concept's ontology identifier to help limit what values are presented to the end-users.
Provider


Property form_comments
Description Comments that appear on the form and describe the schema concept.
Provider


Property tool_tip
Description Text that appears when end-users let their mouse cursor hover over a field label.
Provider


Property record validation services
Description Collection of descriptions of record-level validation services.
Provider

Back to properties list


DataFieldModel Properties

Property name
Description Name of the field.
Provider
  • XML Schema Reader: <xs:element name="[name]" .../> for edit fields;
  • XML Schema Reader: <xs:element ref="[name]" .../> or <xs:group ref="[name]".../> for list fields.Note that for list fields, the name is the record class name of another record structure or the name of a group of record class names.


Property isRequired
Description Determines whether a field is optional or required.
Provider
  • <...minOccurs="0".../> means the field is optional.
  • <...minOccurs="1".../> means the field is required.


Property helpLink
Description The URL for a web page that describes the form concept.
Provider


Property fieldViewType
Description Gives an indicator of how the field should be rendered.
Provider
  • type="xs:string" indicates the field type is TEXT_FIELD.
  • type="xs:date" indicates the field type is DATE_FIELD. The field label will be followed by the date format pattern enclosed in parentheses.
  • If a field has at most three restriction values, the field type will be a RADIO_FIELD. The field will be rendered with radio buttons. If there are more than three restriction values, the field type will be COMBINATION_FIELD. The field will be rendered with a dropdown list of choices.
  • type="xs:anyURI" indicates the field type is URI_FIELD. The field will be rendered with a browse button that allows end-users to search for a file.
  • <xs:attribute.../> indicates the field type will be an ID_FIELD. In the main Project35 form, attribute fields are shown first, followed by all the other fields. An ID_FIELD will also have a "Generate Key" button that users can press to generate an identifier value;
  • <xs:element ref=".."...maxOccurs="1"../> indicates a field will be ONE_TYPE_ONE_VALUE_LIST. This form field will have a desensitised text field and have a "New" and "Edit" buttons;
  • <xs:element ref=".." ... maxOccurs="unbounded"../> indicates a field will be ONE_TYPE_N_VALUE_LIST. This form field will have a scrollable list showing display names of sub-records. It will also have "New", "Edit" and "Delete" buttons;
  • <xs:group ref=".."...maxOccurs="1"../> indicates a field will be a N_TYPE_ONE_VALUE_LIST. The form field will have a desensitised text field and a combination box that lets the user choose which type of record to create;
  • <xs:group ref=".." ...maxOccurs="unbounded"../> indicates a N_TYPE_N_VALUE_LIST field view type. The form field will have a scrollable list showing display names of sub-records. It will also have a combination box that lets the user choose which type of record to create. When users select a type of record, the list filters to show records of that type.


Property ontology_identifier
Description A unique identifier that can be associated with a schema concept. This is useful if ontology services want to use a schema concept's ontology identifier to help limit what values are presented to the end-users.
Provider


Property form_comments
Description Comments that appear on the form and describe the schema concept.
Provider


Property tool_tip
Description Text that appears when end-users let their mouse cursor hover over a field label.
Provider

Back to properties list


EditFieldModel Properties

Property defaultValue
Description The default value that should be displayed whenever a new record containing this field is displayed.
Provider


Property allowFreeText
Description Determines whether a text field entry can accept free-text entries; this is not applicable for fields that have drop-down lists or radio buttons.
Provider


Property scrollingTextField
Description Determines whether a text field is displayed with one line of text or a scrolling text area.
Provider


Property editFieldValidationServices
Description Validation services that can be applied to the value held by the edit field model.
Provider


Property isDisplayNameComponent
Description Determines whether the field is used to derive the name of the containing record.
Provider


Property units
Description The units associated with a field; typically only of use for numeric fields.
Provider


Property fieldValidationServiceConfiguration
Description A collection of descriptions of field validation services.
Provider


Property ontologyServiceConfigurations
Description A collection of descriptions of ontology service descriptions; this is only applicable to text fields.
Provider


Property editingComponentClassName
Description The class name of an editing component; this should desensitise the part of the form field that holds a value. Instead, users click on "Edit" to invoke a separate editing component.
Provider

Back to properties list


IDFieldModel Properties

Property idGeneratorService
Description Generates an identifier value that Project35 inserts into the text field whenever the "Generate Key" button is pressed.
Provider

Back to properties list


GroupFieldModel

Property choices
Description The choices provided in a drop down list.
Provider
  • XML Schema Reader:
    <xs:restriction base="xs:string"> 
         <xs:enumerationvalue="value2"/>
    </xs:restriction>
    

Back to properties list


BooleanFieldModel

Property choices
Description The choices provided in a drop down list.
Provider
  • XML Schema Reader: type="xs:boolean"
  • BooleanFieldModel, a subclass of GroupFieldModel, forces the choices to be "true" and "false".

Back to properties list


ListFieldModel Properties

Property fieldValidationServiceConfigurations
Description A collection of list field validation services.
Provider


Property listFieldEditingComponentConfigurations
Description A collection of descriptions of components that can create or edit different kinds of records in a list field.
Provider

Back to properties list


Design Considerations During the early development of the code base, there was a great temptation to make the application hold all its data in DOM objects. This was because the main I/O routines were written using the DOM parser and the result of the activity was a complete in-memory tree of DOM objects. There could have been some benefit in having Project35 rely on a generic data structure that was used in other projects. However, the DOM object model had shortcomings. The generic data objects had generic means of accessing and changing data. The task of finding specific fields within a record was cumbersome and required a great deal of looping constructs. The cumbersome nature of the DOM API provided motivation to develop a collection of native data structures that could hold better cater for operations supported by the application.

This proved to be a good decision because the application later failed to perform satisfactorily when it loaded large data files. Performance improved dramatically when the file reading routines switched from using the DOM to SAX parser. The SAX parser did not produce DOM objects, so it would have not made sense for the rest of the code base to depend on the generic data structures.

Initially the native data structures held information about both the model and view aspects of records and fields. This later caused severe performance problems because Swing-based field views were created for each field whether they were being viewed or not. The structures were later reworked in a way that rigidly separated model and view aspects. View components were only ever generated for fields in the currently displayed record.

The next problem with the native data structures related to serialisation. The copy and paste feature in the application required that all objects and the objects they referenced are serialisable. This worked well until serialisation encountered things like ValidationService and OntologyService objects. These were interfaces that could be implemented as Java classes that themselves were not serialisable. Eventually, native data structures were stripped of most of references to services so that serialisation had fewer dependencies. The only remaining service that exists as an artefact is the way EditFieldModel references validation services. This legacy feature will be eliminated when the schema reader is improved.

In the revised approach, the form generation facility would receive a form field, look up properties in the configuration reader, and instantiate services when the fields were actually displayed on the screen.

Still, many of the native data structures had too much code. Many of them such as RecordModel had a number of utility methods which did things like identify different groups of fields. This functionality was migrated to a new class called RecordModelUtility.

The result is that the data structures now hold mostly data and not information about views and services. The exception is how DataFieldModel knows about a FieldViewType. This type is something that is determined by the schema reader, and is used to encode rendering hints into the model object. For example, the schema reader can tell whether a list field can support one or multiple types of sub-records. It marks the model object with field view types such as "ONE_TYPE_ONE_VALUE_LIST", and "N_TYPE_ONE_VALUE_LIST". Project35's project35.desktopDeployment.FieldViewFactory uses this field type value to determine what kind of form field to create for visualising the field data.



Scope of Effect Project35's native data structures are used ubiquitously in all tools in the Project35 suite.


Relevant Code Packages All of the model classes are defined in the project35.mda.model package.

Back to top


Record Model Factory

Purpose To create instances of template records that have been defined by the Model Reader System.


Description The Record Model Factory is used to instantiate instances of template records which have been defined by the Model Reader System. The instances are used for two purposes. First, the instances can be used to hold form data created by an end-user. Second, different parts of the application, such as the Form Generation System, can interrogate them to determine application properties associated with concepts defined in the XML schema.


Design Considerations Early in the development of the software, there was a need to centralise the creation of container objects used to hold form data.


Scope of Effect The Record Model Factory is used throughout the code base, both to create new records and to inspect the properties of template data structures that have been synthesised by the Model Reader System.


Relevant Code Packages The Record Model Factory is a single class project35.mda.model.RecordModelFactory.

Back to top


I/O System

Purpose To store and retrieve form data managed by Project35.


Description

Project35 normally stores a data set as a zipped file ending in a *.PDZ file extension. The zipped file contains a number of XML files, each of which represents a layer of information. Currently there are two layers: the data layer and the meta-data layer. The data layer is represented by the *.PDR file and contains the text that would appear in form fields. The tags found in the data layer will be defined in the target schema used to drive the data entry application.

The meta-data layer is represented by the *.META file and contains meta data about the data set, including basic information about the author and about all the ontology terms which were used to mark-up form fields. The tags found in the meta-data layer are defined in the schema: ./models/Project35_meta_data/model/Project35MetaData.xsd.

The I/O system for creating *.PDZ files can be extended to include other information layers (see section Architecture Extensions)

Project35 can export a data set as an XML file that will only contain information from the data layer. This export feature appears in the "Export to Final Submission Format" menu option but will probably be relabelled something more appropriate in the future.



Design Considerations

Use of Layers

Originally, Project35 stored a data set as a single XML file. The need to store a data set as a collection of layers arose from the development of ontology services. Initially, ontology services provided text phrases that would be pasted into forms. However, an ontology term is not adequately represented by a word phrase. Ontology terms were eventually redesigned to use a human-readable label and a machine-readable identifier.

Although the labels for selected ontology terms were stored into form fields, Project35 needed some mechanism for storing information about the unique identifiers. Initially, ontology terms were written as hyperlinks in the XML data file. They were stored in the form label.

The problem with this approach was that data sets marked up with ontology terms would fail to validate against the XML schema. This was because the schema would not describe the '<a>' tag which appeared within the tags for a form field. Rather than treating '<a>' as a tag with special significance, I decided to store ontology term identifiers in a separate meta data file that would accompany the data file. Project35 was modified so that its data sets were stored in ZIP files that contained multiple information layers.

The Structure of Project35’s Native Format *.PDZ File

It contains a *.PDR file which holds the form data and a *.META file which holds the meta data about the data set. The data held in the *.PDR layer validates against the XML Schema and the data held in the *.META layer validates against the meta data schema defined in .\models\project35_meta_data\model\Project35MetaData.xsd.


Changing Parsers

The software used to rely entirely on the DOM parser and still uses it for parsing the meta data file. The DOM parser works by parsing an XML file and producing an in-memory tree of DOM model objects. The API for DOM objects made it easy to extract information from the XML file. The parser performs well with small data sets but exhibited performance problems when it was used to process large data sets. This is because the parser loaded an entire XML file into memory before the DOM objects could be used. The application experienced great performance gains in reading files when some of the I/O classes began using the SAX parser.

Support for Streams

I/O files were modified so they could accept data streams instead of just files. This was done to make it easier to deploy Project35 as a component rather than as a standalone application. In a component mode of activity, Project35 may receive its data input as a stream coming directly from another component.

Creating the "Export to Final Submission" Feature

The software used to allow end-users to export native format *.PDZ files to *.XML files that only contained the data layer of information. The *.XML files tended to be candidate files for submission to data repositories. To ensure users submitted complete, valid data sets to repository managers, the feature "Export to Final Submission Format" was added. When users press this button, it attempts to export the document to an XML document which represents a final draft. If the document fails validation, no final draft is produced. This behaviour was supported to encourage end-users to make an earnest attempt to fix errors before they submitted data files to repositories.

Providing Support for the Meta Data Layer

For most of its development cycle, Project35 has saved an arbitrary collection of meta data in the *.META layer. Typically this focused on recording which ontology terms were used to tag a particular kind of schema concept such as a form field or record.

Whenever a new attribute was added, it resulted in changes made to special I/O routines which read and wrote meta data records. In 2007, the *.META layer was given its own distinct schema for meta data. Each Project35 tool now loads the project35_meta_data model and uses a special context variable (See Accessing Application Variables) to help read and write meta data records. These records are maintained independently of the form data end-user edit through the normal use of the tool.

A new utility has been designed which will allow data curators to edit just the meta data layer of a given *.PDZ file. The Project35 Meta Data Editor uses the same "project35_meta_data" model but allows curators to post-annotate a *.PDZ file. Curators can now remove ontology terms that were used to tag records and fields. Alternatively, they can add more terms using the same ontology services that are available to end-users.

With the new support for the *.META layer, data curators can change the meta data about a data set without editing the data themselves. The layers can be maintained completely independent of one another.



Scope of Effect

Most of the I/O classes are defined in project35.io package. Whereas the meta data used to be managed by project35.io.MetaDataReader and project35.io.MetaDataWriter classes, meta data records are now written using the normal Project35DataFileReader and Project35DataFileWriter classes respectively. Most of the I/O packages are called in the project35.desktopDeployment.FileMenu or project35.tabletDeployment.FileMenu classes.



Relevant Code Packages

The I/O classes appear in project35.io. Project35DataFileReader/Writer are used to manage the .PDR files that represent the data layer of each data set. NativeDataFileReader/Writer uses these classes when it manages the zipped .PDZ files. XMLSubmissionFileReader/Writer wraps Project35DataFileReader/Writer and produces *.XML files.

Back to top


Form Generation System

Purpose To generate UIs for Project35 which suite displays on desktop and Tablet PCs.


Description

Project35's architecture maintains a rigid separation between the structures that hold data and the structures that present them to an end-user. The model aspects are represented by the Native Data Structures, information for which can be found in the Core Architecture section. The view aspects are represented by the following packages: project35.desktopDeployment and project35.tabletDeployment.

General Classes for Generating Desktop Project35 Forms

When end-users invoke the "run_project35" script, it spawns an instance of Project35 Application. The object prompts end-users to load a model and it produces template record definitions that can be used to populate a document.

  • Project35Application creates an instance of an empty Project35Dialog, which is the main window for the data entry tool;
  • Project35Dialog has three major UI components: Project35MenuBar, NavigationTree, and RecordView;
  • Project35MenuBar containing a variety of menus subclassed from Project35Menu;
  • Project35Menu contains general-purpose code for handling plugins;
  • NavigationTree is the tree display that appears on the left part of the window. This manages a tree of NavigationTreeNode objects that mirror the tree of RecordModel objects, which comprise the document;
  • RecordView comprises a RecordViewTitle, which appears flushed left at the top of the panel, and a collection of DataFieldView objects, which render form fields;
  • DataFieldView objects all use a corresponding DataFieldModel object that is part of the currently selected RecordModel object.

General Class Relationship Diagram for Generating Desktop Project35 Forms


Classes for Generating Edit Fields in Desktop Project35 Forms

The image below describes a more detailed view of UI classes that are used to render text fields, identifier fields, combination box fields and radio fields. EditFieldView contains code which can render properties of a corresponding EditFieldModel object. It is responsible for rendering a "Plugins" button for any form field that has been associated with a collection of plugins. The other field view classes use properties defined in corresponding edit field model objects. RadioFieldView uses the choices provided by GroupFieldModel to render a group of radio buttons. If GroupFieldModel provides more than three choices, Project35 uses a CombinationFieldView to render the items as a drop-down list. URIFieldViews use TextFieldModel objects whose XML Schema definitions used "xs:anyType" for the type attribute (See EditFieldModel Properties). IDFieldView uses an instance of IDFieldModel to render a text field that is accompanied by a "Generate Key" button. The identifier service used to provide an identifier is a property of the IDFieldModel.

Class Relationship Diagram for Rendering Edit Fields in Desktop Project35


Classes for Generating List Fields in Desktop Project35 Forms

  • ListFieldView comprises a ListTypeManager, a ListValueManager and it uses an instance of ListFieldModel.
  • ListTypeManager manages the type of child record that will be created or edited when end-users press "New" or "Edit" buttons. There are implementations of ListTypeManager that accommodate lists that support one or multiple types.
  • MultiListTypeManager renders a combination box populated with the kinds of records that can appear in the list field.
  • ListValueManager is an abstract class that manages the display of list items.
  • SingleListValueManager represents a one item list as a single non-editable text field that shows the display name of the child record.
  • MultiListValueManager shows the child records in a scrollable list of record names.
  • project35.desktopDeployment.RecordViewFactory creates list fields. It determines the type of list field to produce by inspecting the fieldViewType attribute of the ListModel object.

Class Relationship Diagram for Rendering List Fields in Desktop Project35


Classes for Generating Forms in Tablet Project35

Tablet Project35 was designed to support the same data models as those used to run Desktop Project35. However, we envisioned that different forms of deployment would be used at different stages of editing the same document. End-users will tend to use Tablet Project35 to do simple data entry tasks which require them to be at a work site such as a laboratory or a remote field location. They will tend to use Desktop Project35 to support complex data entry tasks and to fill in the rest of the document.

Tablet Project35 was designed with the following principles in mind:

  • Minimise the feature set of the application to support essential tasks;
  • Minimise the use of pop-up dialogs;
  • Economise on screen real estate;
  • Change features which rely on right-menu clicks.

The classes used to generate forms in Tablet Project35 are in the project35.tabletDeployment package. Many of them share the same name as other classes that cater to supporting Desktop Project35. There are a few notable differences.

First, some of the menu items have been removed. For example, the File Menu does not contain an "Export to Final Submission Format" button. This feature was deemed non-essential because it was likely this would be done with the Desktop version.

To reduce the number of pop-up dialogs, some features were altered so they used a stack of screens instead of separate windows. For example, consider the "Window" feature in Desktop Project35. When end-users switch from one file to another, a new window grabs focus. In Tablet Project35, only one file is shown at a time. When end-users change the current file, it changes the file loaded in the current window.

As another example, the DefaultOntologyViewer was altered so it relied on JPanel objects instead of JDialog objects. This was done so the ontology viewer was more easily embedded in a stack of windows.

To economise on screen real estate, the NavigationTree was removed. It is accessible via the "Where Am I" button, which pushes a view of the NavigationTree onto the stack of windows. Tablet Project35 supports navigation via a RecordStack object. RecordStack is a drop down list that shows the currently edited branch of the tree.

Finally, the right click mechanism for activating ontology services is replaced by pressing a "Mark-up" button which appears at the end of a text field.



Design Considerations

Initially, Project35 tightly coupled data objects with the objects that viewed them. This led to severe performance problems because Java would potentially be managing thousands of UI components, each using a collection of Swing objects. This led to stripping the native data structures of references to UI components. Eventually the production of form field views was centralised in the class project35.desktopDeployment.RecordViewFactory.

The most significant change in the form generation classes came when Tablet Project35 was developed. Initially we wanted to run the software on a PDA. However, these devices often require a specialised form of the JVM which does not support Swing components. Although the native data structures were stripped of references to UI components, they still had one crucial reference to the swing libraries: the ChangeListener class. We realised it would not be easy to port the code base to a PDA platform so instead we decided on a Tablet PC platform.

It became clear that we had to make some changes to the forms. The NavigationTree was taking up too much room in the display and it was difficult to tap a pen on some of the nodes. Too many windows popped up and they cluttered the screen area. We also observed that it was difficult to activate ontology services. In the Tablet display, a right-click action is done by tapping and holding the pen on a form label. This seemed too awkward to do, so a "Mark up" button was developed instead.

The development of TabletProject35 took 3 consultation meetings with mass spectrometer scientist Jennifer Lynch and ten business days of coding. The result proved that Project35's design isolated its model aspects enough to develop new ways of visualising them.



Scope of Effect The UI classes are probably not used by anything other than classes in Desktop and Tablet deployment packages.


Relevant Code Packages The classes used to generate user interfaces in Project35 are in the project35.desktopDeployment and project35.tabletDeployment packages.

Back to top