The Core of Project35 is the minimum needed to run the application. With this, you can generate forms for immediate use either for data entry or for requirements gathering. To extend the functionality and capability of the core, read through the other sections of the architecture and the rest of the design manual.
Model Reader System |
Schema Reader |
Configuration Reader |
Configuration Tool |
Native Data Structures | Record Model Factory | I/O System | Form Generation System
Each section is structured to address purpose, description, design considerations, scope of effect, and relevant code packages.
Inputs to Project35 come from an XML Schema and a Configuration File. The XML Schema can be produced with any editor you wish to use. The Configuration File is the product of the Project35 Configuration Tool. Both of these inputs provide information for the generation of data entry forms.
The outputs of Project35 are forms that are automatically generated from the model provided at the beginning. These forms are accessed by means of one of two deployment applications: a Desktop and a Tablet.
|Purpose||To coordinate the reading of the XML Schema and the Configuration File.|
|Description||This system is a construct for classification of coding areas that are found within the
|Design Considerations||We wanted a system that was capable of supporting a replacement of the Schema Reader. This can be accomplished by having the rest of the system interact with an abstract interface rather than a specific implementation something that interprets schemas. We needed a system that could extract information about data structures from the schema and another thing to extract information about form properties that were related to that schema. This system's role as a whole is to obtain enough information/properties to drive form generation.|
|Scope of Effect||Nothing else uses this as it's an abstract concept but parts of it will be used in other places as in the case of the Configuration Reader. The components of the Model Reader System are used to synthesise Native Data Structures that are then used by the rest of the program. Most of the rest of the programme will only use the Configuration Reader.|
|Relevant Code Packages||
Back to top
The schema reader sub-system combines information from the XML shema and the Configuration File to create templates of native data structures. These structures are instantiated whenever a new record is needed by the generated data entry tool.
The application relies on an implementation of a SchemaReaderInterface, which is responsible for transforming XML schema structures into native data structures that the rest of the program uses. Currently the
The XML Schema is used to express properties such as the data types of edit fields, the cardinality of list fields and whether fields are required or optional. There are many other application properties supported by the application which cannot be expressed in the XML schema language. The architecture maps records and fields defined in the schema with properties defined in a configuration file. This configuration file is edited via the Project35 Configuration Tool. The properties are interpretted via the Project35ConfigurationReader class.
The schema reader is responsible for combining the information provided by the schema and the properties provided by the configuration reader to create template definitions of data structures that hold data. These native data structures are treated as factories, and are used whenever the users load or create new records for their data sets.
The schema reader is used once to interpret the XML schema and the configuration file created for each data entry model. After it finishes its activity, the rest of the application is driven off the record factories. The native data structures are used for two purposes:
Historically, the template record structures contained many properties which were not necessary to be known by every instance. For example, designers may associate the help file "PhoneNumber.html" with the "phone_number" field of the record "contact". It is not necessary for each instance of the contact record to know about the HTML help file.
The unecessary references to properties took up memory space and caused problems when the data objects were serialised during copy/paste operations. Unfortunately, if you're serialising an object, all of the objects it uses must also be serialisable. In early versions of the code data fields held a reference to instances of services defined in the configuration file. When the field was serialised, the application generated errors because implementations of the services were not serialised. To avoid this problem, the data structures were stripped of many properties that were later managed through a lookup feature in Project35 Configuration Reader.
In activities such as form generation, the application now uses the record type and field name of the current form object to look-up the HTML file that should be made available to end-users.
|Scope of Effect||
The Schema Reader system is only used once when an application
starts. Almost all the application code relies on the templates of
record definitions managed by
Replacing the schema reader would require rigorous unit testing because of the potential for errors when the schema reader uses model properties to set attributes in the template record definitions. The activity lends itself to automated testing and will not require testing of the application itself. For example, test cases could run the schema reader and then perform JUnit test cases that verify that records have the expected number of type of fields.
|Relevant Code Packages||
There are three relevant packages that support the activity of reading the XML schema and its associated ConfigurationFile.xml:
Startup provides the SchemaReaderInterface with the name of the
XML Schema file typically found in
Currently, Project35 uses
Configuration file options are stored in classes whose names end in
"Configuration", such as
Back to top
|Purpose||To manage options for configuring a data entry application that are not expressed in the XML schema.|
Project35 interprets the XML Schema to determine properties of the
data entry application. However, many of the configuration options
canít be expressed in the XML Schema language, so they are managed in
Data modellers use the Project35 Configuration Tool to produce the file "ConfigurationFile.xml", which describes all the application properties that are associated with XML Schema concepts. When the Project35 application starts, it reads the configuration file and holds the information in instances of data container classes. These classes have properties which are analagous to classes defined in the XML Schema for the Project35 Configuration Tool (See the XML schema for the Configuration tool in
Various parts of Project35 use the Project35 Configuration Reader to obtain
configuration data that are used to render the application. For
Other Configuration Files
Project35 maintains two other configuration files in the config directory of a model folder: SessionAspects.xml and FileExtensionsToLaunch.xml
The SessionAspects.xml file contains information about the most recently accessed files, and is managed by the class
The FileExtensionsToLaunch.xml file associates file extensions with shell commands used to launch other software applications. The mappings are used when users press the "View" button on a URL Field View. If the file specified in the text field ends with an recognised extension, Project35 will try to launch an application by making a system call.
It remains the only configuration file that end-users still have to configure. Currently, the
In early incarnations of the software, data modellers had to craft the ConfigurationFile.xml file by hand. The file was awkward and time-consuming to maintain, especially when large XML schemas were being used. This led to the idea of using Project35 to edit its own configuration files. The Project35 Configuration Tool was developed using a schema of configuration properties and a collection of plugins which helped data modellers fill in the forms.
The advent of the Project35 Configuration Tool meant that a configuration file could be designed far more rapidly than it could in previous releases. Enshrining configuration options in an XML Schema also meant it was easy to add new properties.
The configuration options for Project35 expanded to include the selective inclusion of menu items and services which could be associated with a field, record or document scope of effect. With new features came data container classes that held the property values in memory.
|Scope of Effect||
The Configuration Reader is referenced in the file menu classes to determine what features should be included. Many of the classes which generate form fields interact with the
|Relevant Code Packages||Code for
Back to top
|Purpose||The Project35 Configuration Tool provides application designers a graphical application for setting the values of configuration options in generated data entry applications.|
Project35 generates data entry tools based on the record and field concepts specified in an XML Schema. The XML schema language is able to describe properties of form fields such as:
However, the schema is not expressive enough to describe all of the application properties that are supported by a data entry application. For example, there is no facility in XML schema which could be used to associate a context help file with a given record or field.
Features that cannot be inferred from the XML schema are managed in a
configuration file which has a path of
ConfigurationFile.xml stores configuration data for features including:
The ConfigurationFile.xml file also manages the links between parts of the generated applications and services which include Ontology Services, Validation Services, General Purpose Plugins, and ID Generator Services.
The properties of
Architecture of the Project35 Confguration Tool
The above image presents a simplified version of the system described in the main Architecture section of this manual. The software uses the following two files to generate the Project35 Configuration Tool:
The above image shows the various kinds of services which were developed to provide configuration features. Developers should find good examples of how to make services by examining the class files that were developed to support these features.
General Purpose Plugins
The following services implement the
The Configuration Tool relies on an ontology service which uses
Validation ServicesThe Configuration Tool uses two document-level validation services:
Originally this ConfigurationFile.xml was edited by hand. However, the options in the file can be defined in an XML Schema. The software was modified to use its own form generation engine to create a graphical Configuration tool for application designers.
Initially, the configuration files were coded by hand. Over many years, it became clear that manually producing the ConfigurationFile.xml was an obstacle to widespread use of the software.
Using Project35 to produce a Configuration tool for Project35 provided a number of benefits. First, it helped reduce the amount of effort spent coding, testing and documenting a new utility tool. Second, using the software to build its own Configuration tool provided an opportunity to test two things:
Finally, the design supports future extensions of the Configuration Tool. See Architecture Extensions to understand how easy it is to make Project35 support more configuration options.
|Scope of Effect||
Adding, changing, or deleting the plugins will impact two aspects of development:
|Relevant Code Packages||
All of the code for the Configuration Tool relates to the plugins it
supports. The code for plugin features resides in the
Back to top
|Purpose||The native data structures are an in-memory representation of an XML data set.|
The basic native data structure in Project35 is the
Inheritance Hierarchy for Data Field Classes Supported in Project35
All fields will have properties such as:
The two kinds of fields are
There are four kinds of
Aggregation Relationships for Project35 Native Data Structures
The most popular utility class in the
There is a strong relationship between properties of the XML schema and properties of the native data structures. The following tables list the attributes of native data structures that should be set with values derived from a schema:
|Design Considerations||During the early development of the code base, there was a great
temptation to make the application hold all its data in DOM objects. This was because the main I/O routines were written using the DOM parser and the result of the activity was a complete in-memory tree of DOM objects. There could have been some benefit in having Project35 rely on a generic data structure that was used in other projects. However, the DOM object model had shortcomings. The generic data objects had generic means of accessing and changing data. The task of finding specific fields within a record was cumbersome and required a great deal of looping constructs. The cumbersome nature of the DOM API provided motivation to develop a collection of native data structures that could hold better cater for operations supported by the application.
This proved to be a good decision because the application later failed to perform satisfactorily when it loaded large data files. Performance improved dramatically when the file reading routines switched from using the DOM to SAX parser. The SAX parser did not produce DOM objects, so it would have not made sense for the rest of the code base to depend on the generic data structures.
Initially the native data structures held information about both the model and view aspects of records and fields. This later caused severe performance problems because Swing-based field views were created for each field whether they were being viewed or not. The structures were later reworked in a way that rigidly separated model and view aspects. View components were only ever generated for fields in the currently displayed record.
The next problem with the native data structures related to
serialisation. The copy and paste feature in the application required
that all objects and the objects they referenced are serialisable.
This worked well until serialisation encountered things like
In the revised approach, the form generation facility would receive a form field, look up properties in the configuration reader, and instantiate services when the fields were actually displayed on the screen.
Still, many of the native data structures had too much code. Many of them such as
The result is that the data structures now hold mostly data and not information about views and services. The exception is how
|Scope of Effect||Project35's native data structures are used ubiquitously in all tools in the Project35 suite.|
|Relevant Code Packages||All of the model classes are defined in the
Back to top
|Purpose||To create instances of template records that have been defined by the Model Reader System.|
|Description||The Record Model Factory is used to instantiate instances of template records which have been defined by the Model Reader System. The instances are used for two purposes. First, the instances can be used to hold form data created by an end-user. Second, different parts of the application, such as the Form Generation System, can interrogate them to determine application properties associated with concepts defined in the XML schema.|
|Design Considerations||Early in the development of the software, there was a need to centralise the creation of container objects used to hold form data.|
|Scope of Effect||The Record Model Factory is used throughout the code base, both to create new records and to inspect the properties of template data structures that have been synthesised by the Model Reader System.|
|Relevant Code Packages||
The Record Model Factory is a single class
Back to top
|Purpose||To store and retrieve form data managed by Project35.|
Project35 normally stores a data set as a zipped file ending in a *.PDZ file extension. The zipped file contains a number of XML files, each of which represents a layer of information. Currently there are two layers: the data layer and the meta-data layer. The data layer is represented by the *.PDR file and contains the text that would appear in form fields. The tags found in the data layer will be defined in the target schema used to drive the data entry application.
The meta-data layer is represented by the *.META file and contains meta data about the data set, including basic information about the author and about all the ontology terms which were used to mark-up form fields. The tags found in the meta-data layer are defined in the schema:
The I/O system for creating *.PDZ files can be extended to include other information layers (see section Architecture Extensions)
Project35 can export a data set as an XML file that will only contain information from the data layer. This export feature appears in the "Export to Final Submission Format" menu option but will probably be relabelled something more appropriate in the future.
Use of Layers
Originally, Project35 stored a data set as a single XML file. The need to store a data set as a collection of layers arose from the development of ontology services. Initially, ontology services provided text phrases that would be pasted into forms. However, an ontology term is not adequately represented by a word phrase. Ontology terms were eventually redesigned to use a human-readable label and a machine-readable identifier.
Although the labels for selected ontology terms were stored into form
fields, Project35 needed some mechanism for storing information about
the unique identifiers. Initially, ontology terms were written as
hyperlinks in the XML data file. They were stored in the form
The problem with this approach was that data sets marked up with ontology terms would fail to validate against the XML schema. This was because the schema would not describe the '<a>' tag which appeared within the tags for a form field. Rather than treating '<a>' as a tag with special significance, I decided to store ontology term identifiers in a separate meta data file that would accompany the data file. Project35 was modified so that its data sets were stored in ZIP files that contained multiple information layers.
The Structure of Project35ís Native Format *.PDZ File
It contains a *.PDR file which
holds the form data and a *.META file which holds the
meta data about the data set. The data held in the *.PDR
layer validates against the XML Schema and the data held in the
*.META layer validates against the meta data schema
The software used to rely entirely on the DOM parser and still uses it for parsing the meta data file. The DOM parser works by parsing an XML file and producing an in-memory tree of DOM model objects. The API for DOM objects made it easy to extract information from the XML file. The parser performs well with small data sets but exhibited performance problems when it was used to process large data sets. This is because the parser loaded an entire XML file into memory before the DOM objects could be used. The application experienced great performance gains in reading files when some of the I/O classes began using the SAX parser.
Support for Streams
I/O files were modified so they could accept data streams instead of just files. This was done to make it easier to deploy Project35 as a component rather than as a standalone application. In a component mode of activity, Project35 may receive its data input as a stream coming directly from another component.
Creating the "Export to Final Submission" Feature
The software used to allow end-users to export native format *.PDZ files to *.XML files that only contained the data layer of information. The *.XML files tended to be candidate files for submission to data repositories. To ensure users submitted complete, valid data sets to repository managers, the feature "Export to Final Submission Format" was added. When users press this button, it attempts to export the document to an XML document which represents a final draft. If the document fails validation, no final draft is produced. This behaviour was supported to encourage end-users to make an earnest attempt to fix errors before they submitted data files to repositories.
Providing Support for the Meta Data Layer
For most of its development cycle, Project35 has saved an arbitrary collection of meta data in the *.META layer. Typically this focused on recording which ontology terms were used to tag a particular kind of schema concept such as a form field or record.
Whenever a new attribute was added, it resulted in changes made to special I/O routines which read and wrote meta data records. In 2007, the *.META layer was given its own distinct schema for meta data. Each Project35 tool now loads the project35_meta_data model and uses a special context variable (See Accessing Application Variables) to help read and write meta data records. These records are maintained independently of the form data end-user edit through the normal use of the tool.
A new utility has been designed which will allow data curators to edit just the meta data layer of a given *.PDZ file. The Project35 Meta Data Editor uses the same "project35_meta_data" model but allows curators to post-annotate a *.PDZ file. Curators can now remove ontology terms that were used to tag records and fields. Alternatively, they can add more terms using the same ontology services that are available to end-users.
With the new support for the *.META layer, data curators can change the meta data about a data set without editing the data themselves. The layers can be maintained completely independent of one another.
|Scope of Effect||
Most of the I/O classes are defined in
|Relevant Code Packages||
The I/O classes appear in
Back to top
|Purpose||To generate UIs for Project35 which suite displays on desktop and Tablet PCs.|
Project35's architecture maintains a rigid separation between the
structures that hold data and the structures that present them to an
end-user. The model aspects are represented by the Native Data
Structures, information for which can be found in the Core Architecture section. The view aspects are represented
by the following packages:
General Classes for Generating Desktop Project35 Forms
When end-users invoke the "run_project35" script, it spawns an instance of Project35 Application. The object prompts end-users to load a model and it produces template record definitions that can be used to populate a document.
General Class Relationship Diagram for Generating Desktop Project35 Forms
Classes for Generating Edit Fields in Desktop Project35 Forms
The image below describes a more detailed view of UI classes
that are used to render text fields, identifier fields, combination box fields and
Class Relationship Diagram for Rendering Edit Fields in Desktop Project35
Classes for Generating List Fields in Desktop Project35 Forms
Class Relationship Diagram for Rendering List Fields in Desktop Project35
Classes for Generating Forms in Tablet Project35
Tablet Project35 was designed to support the same data models as those used to run Desktop Project35. However, we envisioned that different forms of deployment would be used at different stages of editing the same document. End-users will tend to use Tablet Project35 to do simple data entry tasks which require them to be at a work site such as a laboratory or a remote field location. They will tend to use Desktop Project35 to support complex data entry tasks and to fill in the rest of the document.
Tablet Project35 was designed with the following principles in mind:
The classes used to generate forms in Tablet Project35 are in the
First, some of the menu items have been removed. For example, the File Menu does not contain an "Export to Final Submission Format" button. This feature was deemed non-essential because it was likely this would be done with the Desktop version.
To reduce the number of pop-up dialogs, some features were altered so they used a stack of screens instead of separate windows. For example, consider the "Window" feature in Desktop Project35. When end-users switch from one file to another, a new window grabs focus. In Tablet Project35, only one file is shown at a time. When end-users change the current file, it changes the file loaded in the current window.
As another example, the
To economise on screen real estate, the NavigationTree was removed. It is accessible via the "Where Am I" button, which pushes a view of the
Finally, the right click mechanism for activating ontology services is replaced by pressing a "Mark-up" button which appears at the end of a text field.
Initially, Project35 tightly coupled data objects with the objects that viewed them. This led to severe performance problems because Java would potentially be managing thousands of UI components, each using a collection of Swing objects. This led to stripping the native data structures of references to UI components. Eventually the production of form field views was centralised in the class
The most significant change in the form generation classes came when
Tablet Project35 was developed. Initially we wanted to run the software on a PDA. However, these devices often require a specialised form of the JVM which does not support Swing components. Although the native data structures were stripped of references to UI components, they still had one crucial reference to the swing libraries: the
It became clear that we had to make some changes to the forms. The
The development of
|Scope of Effect||The UI classes are probably not used by anything other than classes in Desktop and Tablet deployment packages.|
|Relevant Code Packages||The classes used to generate user interfaces in Project35 are in the
Back to top