Reporting - Data Ingestion and Analytic Data Schemas

Target release3.4
Epic OLMIS-4637 - Getting issue details... STATUS
Document status
eLMIS StatusUnimplemented
OpenLMIS StatusUnimplemented
Ona Craig Appl (Unlicensed)
OpenLMIS -


Custom reports and, non-custom reports, new reports, and any report in Superset, will need to have the data it depends on available in a database connected to Superset.


Each report requires different sets data to be built, but most of these sets of data overlap will overlap in some way or other.

We can split ingestion the following dimensions

  • Source component (e.g. Requisitions, Stock)
  • Report (e.g. Stock Imbalance, Order Summary)

When we ingest the data and push it into an analytic database we can store it a number of alternative formats (and in multiple simultaneous formats)

  • Replicate in API
  • Replicate structure in underlying databases
    • Could include replicating a structure from a previous version of OLMIS
  • Structure per report
  • Determine higher-level commonalities between repeats and store in those formats


User Stories

#TitleUser StoryLabelImportanceNotesJIRA Tickets

Pull data from APIs

Push data into raw storage (exact same format as when received from API)

Decide on analytic database structure

Push data into analytics databases (this is taken from our raw storage not API)



Jira Epics

OLMIS-4637 - Getting issue details... STATUS

OLMIS-4901 - Getting issue details... STATUS

Open Questions

Below is a list of questions to be addressed as a result of this requirements document:


Gap Estimation Notes 4/12/2018

  • Blocks all reports created in Superset
    • With the caveat that it can be broken down into pieces of ingestion

OpenLMIS: the global initiative for powerful LMIS software