Program Data Technical Design Proposal v2

Overview and Definition

What is Program Data? A working definition:

We call it "Program Data" when:

• a non-programmer would want to change the fields and data collected for implementation-specific needs; and when
• OpenLMIS only needs to know WHETHER the data has been collected, but does not need to know the CONTENTS of what data is collected.

In other words, it is data that is not tied to the OpenLMIS system (its business logic, calculations, notifications). The timing of when this data is received can be asynchronous (collected at a different time).

We may decide to pick a different name. Examples: user-defined data, custom data, non-logistics data, non-stock data, un-actionable data.

For OpenLMIS, there is a certain need to ensure program data has been collected during certain workflows, like requisition.

Requirements and Scope

Requirements for OpenLMIS to support program data, some must have and some nice to have:

Proposal for Program Data Service

In order to create a loose coupling between logistics workflow and program data, there is a need to create a new Program Data microservice. This service is what the rest of the OpenLMIS system (Requisition, Stock Management, Informed Push) would talk to for its program data requirements.

We think that it's sufficient for other services to know whether data has been collected (e.g. to allow submitting a requisition) but not the contents of that data. Therefore we propose a very minimal service that only answers the question "was program data collected for a facility + program + period?"

Of course the program data must be collected, but the other services do not need to know the details of how this happens. Regardless of whether the program data is captured through the OpenLMIS UI, or through an external mobile or web tool, we should mediate all interactions that other services have through this thin service.

A diagram with the interactions:

Open Questions:

• Discussion around the natural key and configuration value or document/source ID.
  • Natural key for requisition service identified as composite of facility, program and processingPeriod, but this may not be true for other services; may need a document/source ID of program data.
  • This natural key may need to be configurable in some way.
• Configuration may be needed on the requisition template (does program data need to be checked for, for this program?) - this is true, but this would probably be dealt with in the requisition service; it does not affect the logic in the program data service

Proposal for DHIS2

We propose using DHIS2 as-is to capture data for facilities, programs and periods, because it is a mature product that already supports this use case. We will do a simple connection that allows OpenLMIS to query DHIS2 and determine whether data has been captured or not.

There are a number of upsides to using DHIS2 as a program data solution.

• It fulfills many of the requirements, including all of the Must Haves, out-of-the-box:
  
  • Form design - can support basic and some complex forms through a WYSIWYG form designer (under Data set, data set management)
  • Data capture - done through Data Entry web app, with some offline support (support for connectivity interruptions measured in seconds/minutes, not hours/days); there is also a mobile web client and an Android app
    • Note: even though the web app does not have full offline support, the fact that many countries with intermittent connectivity are using it, indicates it may be sufficient for program data capture requirements
  • Reporting on data
  • Security - DHIS2 appears to have a robust RBAC design, similar to OpenLMIS
• It has been proven to be usable by country ministries and facilities, and has a community behind it
• There are no dependencies on external development; APIs have been released and are available

The weaknesses of this approach are that if we want it integrated into the OpenLMIS UI, we won't get the benefits that come out-of-the-box. Another weakness is that DHIS2 is another system that must be maintained (although often the country already maintains an instance).

MVP using Program Data Service and DHIS2

As a first step to exploring how program data would work in OpenLMIS, an MVP is proposed. It would have:

• A DHIS2 instance which would handle form design, data capture and reporting as a separate web app and login
• A new Program Data microservice as described in the "Proposal for Program Data Service" section above
  • DHIS2 would be the "backend" for the Program Data Service, in order to answer questions about program data completeness
• A single DHIS2 "OpenLMIS system" user for integration which has the correct DHIS2 permissions to read data required by OpenLMIS 

The new Program Data Service would be a lightweight service which has one API endpoint:

• It would handle requests from consumers–Requisition, Stock Management, Informed Push Services
  • Input: a program data composite (facility, program, processingPeriod)
  • Output: true/false depending on whether that program data is complete
• To answer these requests, it would make an external call to DHIS2 to get relevant data (integration piece)
  
  • Authenticate with a DHIS2 "OpenLMIS system" user
  • Call to DHIS2 APIs, /api/<version>/dataValueSets
    • Input: a mapped program data composite between OpenLMIS and DHIS2
      • An OpenLMIS Facility maps to a DHIS2 Org Unit
      • An OpenLMIS Program maps to a DHIS2 Program
      • An OpenLMIS Processing Period maps to a DHIS2 Period
    • Output: dataValueSet result, with completeDate if program data is complete)

Benefits to doing this MVP with this new microservice and DHIS2:

• It is a low-effort proposal - we think it would take a fraction of the effort of alternative approaches
• We get significant functionality out-of-the-box, including reporting and a WYSIWYG form designer, which we would not get from an MVP using alternative approaches
• It is a low-risk proposal because:
  • We are using a mature tool, so we are not dependent on any external roadmap
  • DHIS2 is widely used, so it is a safe choice for the ecosystem
• We build the Program Data interface and integration to other services
  • The Program Data Service's "backend" (DHIS2) could be swapped out without much waste of effort
  • Additionally, no changes to other Services' logic would be necessary, since the Program Data's API would be unchanged
• It will cheaply test how painful it is to have Program Data in an external system (integrating in the UI will be only of the largest costs of any approach)

Diagram of the interactions:

Component Diagram for the Program Data Service:

Next Steps

Some of this depends on findings in the MVP. However, a list of potential next steps:

• Support for Informed Push - should be straightforward, as it would access the same Program Data endpoint with the same criteria (facility, program, processingPeriod). See interaction diagram above.
• If we like DHIS2 and want to invest more in that integration, then:
  • Explore single sign-on (SSO) or synchronize users
  • Explore integrated "look and feel" - with SSO, it may be possible to put access to DHIS2 in an iframe in the OpenLMIS UI, but effort would be required to give it the same "look and feel"
• If we find DHIS2 is unsuitable, then pivot to supporting ODK2 as a "backend"
• If we find that there is a dependency on the content of the Program Data, then work on returning basic key-value pairs of program data and not just a flag of program data completeness