Architecture Goals

From Design Workshop:

Modularity
Loosely Coupled but Highly Cohesive
Extensions
Backwards Compatibility of Core
Shared Value
Performant at the Last Mile
Scalable to Regional Size
Secure
Deployable w/out custom code
Offline as first class
Easy for New (low capacity) Developers to Start
Easy for contributors to be good citizens with minimal coordination
Architecture supports governance through code

Components

The following component descriptions highlight extensibility by defining how each component contributes, what piece of extensibility it is meant to address and finally how they come together to form a functioning system.

Architecture Diagrams

Detailed OpenLMIS v3 Architecture Diagram.

The following high-level diagram is a simplified view which can be used for presentations and to provide a general understanding.

Extensible through two approaches

Independent Services that communicate through RESTful APIs for different functional areas. E.g. Requisition Service, Informed Push Service, Inventory Management Service, etc
A Spring Context for extension within an independent service. e.g. adding a new suggested order quantity to the Requisition service, adding a new accounting policy to the Inventory Management Service

An Independent Service

A Spring Boot application that runs its own HTTP server and exposes a RESTful API
Extensible through the use of Extension Module(s)
Hosted in 1 + M (1 for Spring Boot and M for the Extension Modules) Git repositories
May expect a PostgreSQL DB connection setting to be provided to it through the environment
Packaged and deployed as an Image or Container

Service

- Defines the interface and default behavior
- May expose Extension Points that an Extension Module may utilize to extend its behavior
- Provides Data Store, logging, and authorization services to an Extension Module
- Hosted in a git repo: openlmis-serviceName

Extension Module

- A Spring Context which uses a Service's extension point(s) to define new behavior
- Packaged and deployed as a JAR through Maven
- Hosted in a git repo: openlmis-serviceName-moduleName

Reference UI Module

- the JS, HTML, CSS, etc web-components that make up the Independent Service's UI. e.g. Requisition
- packaged and deployed separately from the Service
- Hosted in a Git Repo: openlmis-serviceName-refUI

Reference UI

Modular JS framework to compose Reference UI Modules
Packaged as JAR/TAR/ZIP
Hosted in a Git Repo: openlmis-reference-ui

Reference Distribution

Image / Container composition file(s) for wiring Independent Services together and deploying the Reference UI (by launching HTTPd)
Includes a Reverse Proxy and Service Registry so that all of the Independent Services may communicate (Nginx)
Includes script(s) to manage deployment configuration (e.g. PostgreSQL connection, log dump connection, etc)
Hosted in its own Git repository: open-lmis?

Use Cases for Using or Extending OpenLMIS

(copied from https://openlmis.atlassian.net/wiki/x/WYARAw - some Q/A has been removed to start toward being a narrative)

Use Case 1: Project deploys "vanilla" OpenLMIS Reference Distrubution

For this simple case, we want to deploy the reference distribution. No additional modules or enhancements

Seeding data?
Configuration: email, db connection, and other items typically found in default.properties

In this case, the implementer would fork the Reference Distribution repository, configure deployment, and deploy all of the Independent Services and the Reference UI.

For seeding data, since each Independent Service manages its own data in its own logical data store, it is in charge of seeding data. (The physical data store information would be in the Reference Distribution configuration, most likely pointing to an external PostgreSQL database.) There will be some sort of script or task (in each Independent Service) that seeds data.

What about situations where services have dependencies on other services / data being in place? For example, an informed push module may expect that facilities have CCE installed. However without a CCE service being part of the distribution this dependency may not be met.

This dependency issue would show up at the service level, i.e. the Informed Push Independent Service would make calls to the CCE Independent Service, but since it is not installed, those calls would fail. To prevent this, the Reference Distribution would have have dependency management during build/package time. So an Informed Push Independent Service, when it is being packaged, if it depends on the CCE Independent Service, would make sure that is packaged in the Reference Distribution as well.

Each Independent Service will have two build tasks for migration–one for structural migrations and another for implementation-specific migrations. Seed migrations are part of the second type.

Configuration will be managed in the Reference Distribution repository.

Use Case 1a: Project updates "vanilla" OpenLMIS installation to a newer version

A project has deployed the stock OpenLMIS Reference Distribution. Over time, updates were made to the Reference Distribution services, e.g. updates to Requisitions. The project now wishes to update the installation to use the newer services. Must consider DB migration, etc.

In this case, the implementer would get new versions of the images of each Independent Service that needs to be updated. The new images may have structural migrations that need to be run against their respective logical data stores. Implementation-specific (including seed) migrations will not be run on an update.

Use Case 2: Project creates new Independent Service, deploys as part of Reference Distribution

A project enhances the OpenLMIS Reference Distribution with a new service and deploys it. The module is not part of the Reference Distribution, but is published and available for others to include in their own distribution. Use case should include how other projects obtain and leverage this module.

In this case, the implementer would create a new Independent Service image, based on a provided template in a separate repository, most likely forking the repository. They would add any necessary structural and implementation-specific migrations (including seed data) to their own logical data store. If other projects would want to obtain and leverage the module, the original implementer can publish their repository and/or publish their image.

Since the new Independent Service follows the same form as the ones in the Reference Distribution, other projects that use it will run the structural migrations, but ignore the implementation-specific migrations.

If the new Independent Service has a UI component to it, create a new UI module, based on a provided template. Add code to the module to add any new UI necessary. Fork the Reference UI (which is just a framework to compose UI modules), and add the new UI module to its configuration.

They would also fork the Reference Distribution repository, configure deployment to include the new Independent Service, and deploy all of the Independent Services and Reference UI.

Use Case 3: Project extends a core domain model

As an implementer, I want to add several new fields to a Facility record, say "Color", "Max Occupancy" and "Chairs", an n-length list of chairs available at the facility (this is simply non-sense data for this use case). OpenLMIS should define a means to persist these new fields in the database, and retrieve them with RESTful calls, and use them in standard queries (e.g. return all facilities where color == 'blue').

Core domain models, such as Facility, will be accessible in a Core Domain Independent Service. For core domain models that are expected to be extended, an ExtraData property will be provided, where projects can add new fields. Projects will be able to do CRUD operations on the ExtraData property through REST API calls. It has not been determined whether this extra data will be queryable, but possible approaches for ExtraData seem promising for supporting it (e.g.: can query json columns in Postgres).

Use Case 3a: Project updates UI for a core domain extension

A project has extended the Facility record as described in the above Use Case 3. How is the reference Facility UI screen extended to display these new fields? Note the potential data types that the UI might need to accommodate, e.g. "Max Occupancy" as an integer, Color as a color choose or set of RGB values, and Chairs is a selection list widget.

For core domain models that are expected to be extended, their UI component would already have some way to parse the ExtraData JSON for "primitive" types (integer, text, etc.), to display. For types that are constrained by a set of options, the options will need to be provided somewhere(?).

For any UI changes that are more complex, or if the look and feel of the UI needs to be changed for these new fields, the corresponding reference UI module will need to be forked and modified.

Use Case 4: Add a new service -- Informed Push -- as part of Reference Distribution

We want to update the reference distribution with a new capability. For this example, say it is Informed Push

Update reference distribution
- Update docker image configuration to add new docker instance for Informed Push. Will hold image of the new Spring boot application
Starting published Spring boot application template, create new Spring boot application. Write informed push logic here
- If extensibility is foreseen, instead write informed push as a Spring module
UI: create new UI component (starting from published UI template). Update reference UI application to include

This use case is mostly the same as Use Case 2. The main difference is that once the new Independent Service has been created, published and reviewed by the OpenLMIS community, the Reference Distribution and Reference UI would be updated to include the new Independent Service, and those repositories would come under the governance of the community.

Use Case 5: Adding a Strategy to an Independent Service

How does a project insert their own strategy to an OpenLMIS extension point?

Some Independent Services will have extension points, where its Service can be extended by an Extension Module. These extension points will most likely be Java interfaces, where the Service will run a strategy. There will be a default strategy implemented, but can be overridden.

A project can insert their own strategy by creating an Extension Module, based on a provided template in a repository. This Extension Module will have a Java class that implements the extension point interface with a new strategy. The Extension Module is packaged with the rest of the Independent Service, its Spring context will be active in deployment, and the new strategy will be used.

API

A primary design goal of the re-architecture is to expose a published API that all clients of a Service will utilize to communicate with it (i.e. other Services, UI). Services which may only communicate through their published API is a good design practice in enforcing coding to an interface and achieving loosely coupled components. In addition to communicating through published APIs, it is a design goal to not build ...

TODO

Reporting

Reporting needs are generally very implementation specific and OpenLMIS has seen considerable variation between source code in this area. To address implementation needs in customization, OpenLMIS defines two types of reports:

out-of-the-box reference reports
reporting tool set

A limited set of reference reports come with OpenLMIS. Such reports are designed for the general use-case and take the perspective that they would serve as a starting point to be taken and customized in a bespoke manner should they require customization by the implementer. A reference report is one that comes with, and therefore has the perspective of, an Independent Service (IS). By being a part of an IS, they naturally rely on the other ISs that theirs relies on, and would use the same RESTful APIs exposed by the other ISs. Being a reference report, they’d either reside within or adjacent to a Reference UI Module of the IS.

For reports that require heavy customization, require a broad view, or those for which performance is not met by built-in reports, a tool set will be recommended and provided where possible that enables the type of reporting that BI tools are built for. To support BI tool integration, each Independent Service is expected to be able to make available a read-only connection to its data store(s).

A potential integration strategy that would allow for reports to be more easily accessible in services which are meant to be reusable will be to design a feature within OpenLMIS that allows reports as files to be delieverd to users through a file repository type model. This would allow external analysis to be preformed but make the user experience in accessing those reports appear seamless.

Templates

OpenLMIS v3 architecture is complicated where it needs to be to achieve shared benefit. To address this complication OpenLMIS will provide code-level project Templates that will aid in creating new functionality rapidly. These Templates will help the community structure their projects and will include a base set of dependencies and tooling that are recommended. Templates would exist for a Service, Extension Module, and Reference UI Module - the basic set of components that enable modularity. Once a project is started a template wouldn't be involved in migrations or maintenance which allows for projects to evolve their own structure over time as needed. Templates reside in their own repository that anyone may start from.

Service Template

The Service Template defines an empty Spring Boot application that would expect basic application configuration, such as data store or email server credentials, to be provided to it. It would setup naming conventions for URLs, and broadcasting basic versioning information. It would include a basic Gradle build system that sets up basic tasks and shows how Extension Modules would be loaded into the Spring Application Context and found through a Maven repository. It would also help in defining at least two common tasks needed to make services shareable across implementations as well as deployable for implementations: structural data store migrations and implementation's data migrations.

The service template resides in the Git repository openlmis-template-service

Extension Module Template

The Extension Module Template consists of a basic Spring Application Context, as well as a Gradle build system that supports building and publishing to a Maven repository. This template is intended to go along with the Service Template above so that common Spring dependencies are known.

The extension module template resides in the Git repository openlmis-template-extensionModule

Reference UI Module Template

The Reference UI Module Template consists of all of the typical front-end web components: HTML, CSS, JavaScript, etc and a build system that's capable of common tasks such as linting and minification. It also supports packaging for inclusion of the module in the Reference UI.

The reference UI Module Template resides in the Git repository openlmis-template-uiModule

Tech Stack

Service

Java 8
Spring Boot
Spring Data JPA
Docker
Gradle
JUnit
Mockito
PowerMock
WireMock
SLF4J

Extension Module

Spring Context
Gradle
Maven

Reference Distribution

Docker Compose (configure and compose)
Nginx (Reverse Proxy, HTTPd)
Consul (Service Registry)
RSyslog (logging)
PostgreSQL (connection string, composed with docker compose)

UI

AngularJS & RequireJS
Node
NPM
Grunt
Open Web Apps
Protractor

Open Questions

Do we want a Service to be the only one allowed to define a RESTful api? Should extension modules be allowed to? Core Domain would be an example of an Extension Module creating RESTful endpoints, while most other service extension modules would be modifying behavior.
- Is there one Core Domain service, or many smaller services that provide reference data? What is the Service as opposed to an Extension Module of a Core Domain Service?
- Is an Extension Module to a Service able to support new UI and therefore each Extension Module would have a reference UI component in its own git repo?
- Should an Extension Module ever not utilize at least one Extension Point? Wouldn't such a thing really be a new Independent Service?
- Should an Extension Module migrate the data store? Either structure or "data".
Do we want another HTTPd server for the Reference UI, or could we utilize the reverse proxy (Nginx) within the Reference Distribution? Testing?
API Gateway?
Service Registry?
Image or Container tech?
- + Container: known tech with Docker for running, composing, etc