Plugin Architecture Concept Notes

Problem to solve

Customizing and deploying OpenLMIS often involves code-changes. Adding or changing code quickly becomes a communication & synchronization problem: where does a new developer/country start from if they want to add a feature or modify some functionality. The technical side of this problem involves managing:

Source code revision control - Development / collaboration tools such as Git make it easy to track what's been done, however using one repository for a project's entire code-base leads to a problem of where does one start and who needs to approve code additions. e.g. with 2 developers, Alice and Bob, if Alice is designing a stock management feature and Bob is designing a user-security feature, these two pieces are very different pieces of functionality that likely don't depend on one-another, tools such as git aren't really meant to allow Alice and Bob to develop their respective features highly independently.
Database - A database for OpenLMIS is essentially global-state persistent data. Any changes to the schema is problematic as it could effect many places in the code unintentionally and it also introduces a problem of upgrading production systems that are on a legacy schema.
configuration - configuration is clearly often feature-related. Setting up R&R form templates is not related to distribution delivery zones.
localization - localization tags are often clearly feature-related and should be packaged with said features. e.g. tags related to distributions are not very similar to user management.
process flow - the wiring of how a feature(s) is connected can often change. Routine/coverage data is an excellent example of this where we know many countries want to collect some sort of data, however how it's collected, what's collected, how it's approved, etc can change often.

Why an architecture?

A plugin architecture can help alleviate this problem by defining a small immutable core and extension points to that. This is useful as this small immutable core should change infrequently allowing production databases related to that core to be upgraded in a slower more rigorous process. Extension points define ways in which the small immutable core can be used in different ways to extend the core functionality. Such an architecture can ideally support:

Extension points for combining the "core" into new or changed functionality
Independent database manipulations allow extensions of the database to occur so that a plugin's activation/deactivation on a production database is independent of other orthogonal features
Dependency management allows features to be built that specify their dependencies. For example a stock management reporting feature might depend on a stock management feature. The plugins would define this ensuring that when deployed the plugin exists in an environment it can work in.
Plugins are developed in independent git projects so that commits belonging to orthogonal features aren't mingled in the same version control project.
Run-time feature deployment allows a running system's feature set to be expanded/contracted by activating/de-activating a plugin. This is advanced functionality that is more difficult to obtain, but would for example allow a "feature" such as a new report or dashboard to be deployed without having to rebuild the system.

All of these combined allow features to be developed in a more orthogonal style when such features are orthogonal, as well as helps the feature developer know where to draw the line between their feature and core, or their feature and another feature.

Luckily OpenLMIS was implemented with some modularity that gets us many of the above benefits. What's problematic is:

Database manipulations are all mixed together
The "core" needs more thought into what's truly "core" and mostly "immutable" and where appropriate extension points are placed (e.g. regimen data in R&R)
The project is one git project (with minor exception). This is good given the lack of plugin architecture and tooling to help developers along, but means that all feature development is mixed in revision control. Features are left to be pulled apart at the file-level.
The web-components of everything are in one module. While there's some file-structure hierarchy as well as AngularJS design to help separate features here, the web components are all in one place.
The modularization as it exists is a build-time modularization (gradle sub-projects). This makes it easier on the developer to build and deploy, but means that any changes require a full rebuild and re-deployment of the application. This makes deploying lighter-weight plugins/modules such as online reports (e.g. dashboard on stock management) a heavy process.

Immediate (3 months?)

Implementing a full OSGi compliant architecture will not be trivial. In the meantime the following can be done:

configuring flyway db migrations to be a part of each gradle sub-project will help features to be deployed on a DB orthogonally
prototype solution to moving web-components (less, offline, web-services, AngularJS, etc) to be a part of each gradle sub-project to better enhance feature deployment orthogonality
start iterating core-related changes on OpenLMIS/dev for making pull requests to eLMIS/dev so that the two don't get too out of sync before a plugin architecture is implemented
start defining what the small immutable core is, where extension points are best placed
Evaluate existing implementations in more depth: Equinox, Atlassian's. Also evaluate non-OSGi alternatives JSPF, Spring Plugin (though these generally don't get run-time deployment which could be very useful for updating reports) and choose one to implement

Mid-term (1 year?)

In the long term, several steps should be taken to help create this architecture:

Implement chosen architecture in OpenLMIS
likely convert some/all existing gradle sub-projects into plugins. e.g. distribution, email, etc
move plugins into separate git projects
develop template and tooling to help start new plugins as well as test plugins in different combinations

Future

Existing features re-configured & new features are developed as versioned plugins that define their versioned dependencies
Community process exists for changing the "immutable" core when existing plugin extension points aren't capable of meeting plugin implementation demand
Community commons exists for sharing, developing, extending plugins built by community