Background

The Overview of Data Storage Options for OpenLMIS Reporting Platform presented numerous options for storing data in the reporting platform. The team needs to decide the appropriate technology to use for the reference distribution of the OpenLMIS reporting platform that will be used as a starting point for OpenLMIS v3 reporting systems. The document provides a general overview.

Assumptions

The Gap project includes the requirement to build a reporting system that is extensible and able to be run in country. This section provides more details on the reporting system assumptions to support this decision:

• The system must be able to be run within a data center at the target location. Cloud hosting is optional, but not the target.
  • Hardware procurement in many locations is expensive and has a long lead time because systems need to be shipped internationally.
  • In this instance, we must target the "lightest" hardware required to run the reporting stack that can be maintained onsite
  • The team deploying the hardware must have the requisite skills to run the reporting stack.|
    More common skills include:
    • Linux/Unix or Windows Server
    • Web server (nginX, Tomcat, Jetty)
    • Docker
    • RDBMS technologies (MySQL, PostgreSQL, MSSQL)
• Reports are currently written by database engineers in OpenLMIS v2 who are skilled in PostgreSQL. We should give preference to a storage solution that can utilize these SQL skills.
• OpenLMIS is not optimized for streaming data. Reports can be updated on an hourly, daily or weekly basis depending on the deployment and report consumers.

Recommendations

The Ona team recommends that the GAP reporting platform utilize a relational database management system (RDBMS), specifically PostgreSQL, which is currently deployed in OpenLMIS v3. This storage solution requires the least amount of hardware of the three systems and utilizes the current skills that are available in the target deployment areas. If performance becomes a problem for dashboard users, or the core requirements change, the implementations can upgrade to the other technologies, Druid and HDFS.

NB: A key design goal of the reporting platform is that the various components be built modularly. We will design the system so that we can replace the data storage system with something other than an RDBMS in the future with no or insignificant changes to the other components.