Reporting and Analytics
Target release | |
|---|---|
Epic | |
Document status | In progress |
Document owner | @Mary Jo Kochendorfer (Deactivated) and @Clay Crosby (Unlicensed) |
Technical Lead | @Josh Zamor (Deactivated) |
Goals/Scope
The reporting and analytics vision is to support OpenLMIS users with both routine reporting and ad-hoc analysis. Users need a way to extract and visualize data. This specification is not defining the exact list of reports needed but rather what types of reports (map, offline, etc.) and functionality is desired. Please see the Scope Out section for a clear explanation of what pieces this spec does not include like alerts (via email/sms) and integrations to systems like DHIS2.
This wiki page is an attempt to define the reporting and analysis needs for OpenLMIS users based on the following generalized user personas identified by the community.
Background
In an attempt to gain buy-in and agreement by the OpenLMIS community on what the needs, requirements and definitions are for reporting and analytics within OpenLMIS. Do not focus on the technical solution as there may be multiple approaches to meeting the desired behaviors. Focus of this spec is to outline what the end users will need and want. This spec will support the decision making process for implementing reporting and analytics within OpenLMIS.
To support this conversation, we define reporting, dashboards and analytics in the following ways:
Reports, routine reporting and sometimes called built-in reporting contain detailed data in a tabular format and typically display numbers and text only, but they can use visualizations to highlight key data. Some key characteristics of a report include :
It presents numbers and text in a table.
It can contain visualizations (like a map, chart, graph) but only used to highlight findings in the data.
It is optimized for printing and exporting to a digital document format such as CSV, Word or PDF.
It is geared towards people who prefer to read data, for example, lawyers, who would rather read text over interpreting visualizations, and accountants, who are comfortable working with raw numbers.
Rendered from a defined query/template.
Does not require high technical capacity to run or access the report
Usually viewed or run on a routine bases (more than once a year)
Usually drives users to an action
Assumptions:
Designer is an engineer.
Routine reporting applies typically to more than one implementation.
Dashboards are a data visualization tool that displays the current status of metrics and key performance indicators (KPIs) for OpenLMIS end users. Dashboards consolidate and arrange numbers, metrics and sometimes performance scorecards on a single screen. Some key characteristics of a dashboard:
All the visualizations fit on a single computer screen — scrolling to see more violates the definition of a dashboard.
It shows the most important performance indicators / performance measures to be monitored.
Interactivity such as filtering and drill-down can be used in a dashboard; however, those types of actions should not be required to see which performance indicators are under performing.
It is not designed exclusively for high-level decision makers or but rather should be used by the general workforce as effective dashboards are easy to understand and use.
The displayed data automatically updated without any assistance from the user. The frequency of the update will vary by organization and by purpose. The most effective dashboards have data updated at least on a daily basis.
Does not require high technical capacity to access
Assumptions:
Designer is an engineer.
A dashboard applies typically to more than one implementation.
Analytics and Ad-hoc analysis leverages tools that offer the ability to select ad-hoc date ranges, pick different products, or drill down to more detailed data to be dashboards and my dynamic than reports. Some key characteristics of data analytics include :
user can define desired data element combinations (via a programming language or wizard).
Not routine and the combination of data elements is run infrequently (once year).
It fits on one screen, but there may be scroll bars for tables with too many rows or charts with too many data points.
It is highly interactive and usually provides functionality like filtering and drill downs.
It is primarily used to find correlations, trends, outliers (anomalies), patterns, and business conditions in data.
The data used in a ad-hoc analysis is generally historical data. However, there are some cases where real-time data is analyzed.
An outcome of ad-hoc analysis might be to define/identify performance indicators for use in reporting and/or dashboards.
It is typically relied on by technically savvy users like data analysts and researchers with knowledge of and experience with report programming languages.
Assumptions:
Designer needs to be technically inclined to their BI&A tool of choice (e.g. Excel, DHIS2, Tableau, etc). Per-seat licensing for report designer and consumer may apply depending on tool choice.
Typically more implementation specific, less re-use across implementations.
Above definitions were taken and adapted from a DashboardInsights article.
User Personas
User | Tech aptitude | Scope of supervision | Kind of Reports | Access (how/when) | Frequency** | Report Types |
|---|---|---|---|---|---|---|
Store Manager Facility in-charge/administrator | Low | Only responsible for one geographic facility but may have multiple storerooms | Routine reporting |
| Monthly or more (if available) |
|
Intermediate Store Manager (could be district, regional, province) District/regional/provincial health officer, program coordinators, pharmacist |
| Supervises a subset of facilities or specific zone. Facilities must be mapped to zones. | Routine reporting |
| Monthly or weekly depending on replenishment schedules Daily around due dates for requisitions. |
|
Central Personnel or Central Program personnel |
| National supervision of all geographic facilities OR national supervision of one program. | Routine Some ad-hoc (where VAN is rolled out) |
|
|
|
Technical Administrator (can be from the MIS department of the MOH; sometimes there is both a Technical Administrator and a Managerial Administrator) |
| Create new slices and dashboards, templates and conduct ad-hoc reporting. | Ad-hoc requests from MOH, stakeholders ("I want to see...") | Upon request |
| |
Implementer (most likely a contractor/vendor supporting the system) |
| May create new slices and dashboards, or customize and alter existing slices and dashboards based on requests from stakeholders | Ad-hoc Routine (help running or troubleshooting/ |
| (Would use and customize all of the above) | |
Stakeholders outside OpenLMIS | Varies | May represent directorates (public health, any interest programs), stakeholders, partners and donors like UNICEF, GAVI, etc. | Ad-hoc | Reports may be shared with these users outside OpenLMIS as needed, whether on paper or by forwarding an email containing a PDF | ||
**If the implementer is using stock management, frequency of reports would increase.
Assumptions
If the implementer is using stock management, frequency of reports would increase with transaction data.
Project Status - Phase 1
Phase 1 Scope: DISC Indicators 1 and 2 (Full Stock Availability and Closed Vial Wastage) - Last Update 1/16
Load mock data through NiFi, index in s3, and into Druid
NiFi pulls data directly from OpenLMIS from 2-3 API endpoints, flows through Kafka, s3, Druid, to Superset for visualization
Write query for DISC 1 - Full Stock Availability in Superset
Write query for DISC 2 - Closed Vial Wastage in Superset
Data Model
Stock cards - each record / "row" in the database is a stock card line item, and will have the following data elements / fields / "columns" ("row" and "column" are used to show analogous terms to relational databases, whose structure is more familiar, although in this particular database we store the data as .json records)
Stock card line item id ("id")
Stock card id
Occurred date
Orderable name
Orderable id
Orderable code
Commodity type
Trade item
Lot #
Program name
Program id
Program status
Facility supported programs
Quantity
Stock on hand
Reason type
Reason category
Reason name
Source
Facility id
Facility name
Facility type
District
Region
Region code - ISO code
Country
Ideal stock amount - value is added for all orderables within the corresponding commodity type + facility pair
Prioritized List of DISC Indicators & Business Logic for Each
Priority | Indicator Name | Numerator | Denominator | Group Bys | Notes |
|---|---|---|---|---|---|
1 | Full Stock Availability | Count of facilities where, for all line items within reporting period, stock on hand is always > 0. | Count of facilities | Facility, district, province, country, orderable, commodity type, trade item | Defined as absence of stockouts between re-supply periods |
2 | Closed Vial Wastage | Quantity of vaccines that were lost due to Closed vial wastage. "Closed vial wastage" is defined as:
Note: these reason names may vary based upon your implementation's configuration | Total vaccine stock under management during reporting period "Stock under management" is defined as the sum of stock on hand at the start of the reporting period plus the sum of the quantity for all receipts Receipts are defined in one of the following three ways:
| Facility, district, province, country, orderable, commodity type, trade item | |
3 | Stocked According to Plan | Yes/No indicator where "Yes" means the sum of the stock on hand for all orderables within a commodity type is between the Maximum stock level and Minimum stock level for that facility. Maximum stock level = the ideal stock amount for that Commodity Type + Facility Minimum stock level = 25% of the ideal stock amount for that Commodity Type + Facility | N/A - this metric is a count | Facility, district, province, country, commodity type | Because this indicator relies on Ideal Stock Amount, which is only as granular as Commodity Type, this indicator should not be viewed at the orderable-level. |
4 | Forecast Demand Ratio | Quantity of doses consumed "Consumption" is defined as having reason name = "Transfer Out" | # of doses forecasted "Forecasted" = the ideal stock amount | Facility, district, province, country, commodity type | Because this indicator relies on Ideal Stock Amount, which is only as granular as Commodity Type, this indicator should not be viewed at the orderable-level. |
5 | On-Time and In-Full Deliveries | ||||
6 | Functional Status of Cold Chain Equipment | FYI @Vidya Sampath moved this down since we will have a screen to see which CCE is functioning and non-functioning | |||
7 | Temperature Alarm Rates |
Additional detail can be found in the initial design document: DISC Indicators.PDF
Implementation Considerations
The following items will need to be reviewed and potentially customized for your particular implementation's configuration
Product model hierarchy
Based upon the Commodity Types your instance uses, you may need to ensure the data model includes the proper tiers for your instance.
Geo Zone hierarchy
By default, we used the following Geo Zone hierarchy, but you may need to adjust this based upon the hierarchy in place at your instance:
Facility
District
Province / region
Country
Closed Vial Wastage
You should review the definition of line items that count as "closed vial wastage" ("numerator" column for Closed Vial Wastage in business logic table above) to ensure this definition comprehensively applies at your instance.
You should review the definition of "receipts" ("denominator" column for Closed Vial Wastage in business logic table above) to ensure this definition comprehensively applies at your instance.
Consumption
You should review the definition of "consumption" ("numerator' column for Forecasted Demand Ratio in business logic table above) to ensure this definition comprehensively applies at your instance.
User Stories
Title | User Story | Label | Importance | Notes | |
|---|---|---|---|---|---|
| 1 | Infastructure | As an Implementer, I need a way to create reporting indicators based off of OpenLMIS data so that my users can easily see trends and important KPIs. | Vaccines | Must Have | |
| 2 | Report writing | As a Technical Administrator, I need a way to create ad-hoc reports based on OpenLMIS data already mapped to Superset so that I can create new slices (charts, graphs, etc) based feedback from users. The reporting platform comes with a predefined data model based on stock card and requisition data (future), and this data model was architected with these DISC indicators in mind. Note: if your deployment has additional indicators you would like added that require data elements not included in the current data model, please contact your OpenLMIS representative. | Vaccines | Must Have | |
| 3 | Full Stock Availability | As an Intermediate Store Manager, I need to see the full stock availability for my supervised facilities so that I can effectively reorder stock to reduce stock outs at my supervised facilities. | Vaccines | Must Have | |
| 4 | Closed Vial Wastage | As an Intermediate Store Manager, I need to see the closed vial wastage for my supervised facilities so that I can identify high levels of wastage to figure out where focus on improving the vaccine ordering, distribution and storeroom management. | Vaccines | Must Have | |
| 5 |
OpenLMIS: the global initiative for powerful LMIS software