/
Team Ona Release 3.5 Backlog Grooming Brainstorming

Team Ona Release 3.5 Backlog Grooming Brainstorming

Aug 22, 2018

This brainstorming was originally completed in Aug 2018

End state for the 3.5 Release Reporting Stack (unordered):

  • All Reports in the Gap Roadmap have been scoped and mapped against the OpenLMIS v3 APIs

    • This includes evaluating whether OpenLMIS v3 has the features available to generate each report

    • A list of reports that can't be created due to feature discrepancy has been published to manage expectations

  • A list of candidate endpoints for data pumps has been created to inform the development of those features.

  • Multiple teams have experience with and are actively developing reports in the reporting stack

  • Automation of the end-to-end reporting stack has improved minimizing the amount of human intervention required

  • CI/CD processes for the reporting stack have been developed, documented and tested for at least 1 report

  • The Nifi Registry supports TLS and is actively being updated by developers as the flows are developed

  • An evaluation has been made for the Single Sign-on needs and capabilities and a go/no-go decision has been made

Epics:

We want to move to smaller sized deliverables by creating an Epic that can be achieved in one or two sprints. We would like to get to a point where the backlog grooming is focused on scoping an Epic. For example, these N epics together will get us to the bigger goal of the 3.5 release.

(Priority: Medium) Improve Automation of loading Superset https://openlmis.atlassian.net/browse/OLMIS-5270 
When you create a chart in Superset, each chart has a different configuration. If it's a bar graph, you need to know the configuration that's specific for Bar Charts in the JSON that we define in Nifi. We can solve this by adding JSON in Nifi per chart type.
Auto loading the data source, we have to create the data source in Superset and then create the .yaml file
(Priority: High | Antonate) Ingesting referencedata microservice and store in OpenLMIS v3 postgres data structurehttps://openlmis.atlassian.net/browse/OLMIS-5266
(Priority: High | Clay) Perform the following for the Reporting Rate Report

(template) Mapping from each report to data table required for that report, and from that to  the API endpoints storing that data

(Priority: High| Jason)Reporting Jenkins https://openlmis.atlassian.net/browse/OLMIS-5257
Standing up Jenkins jobs for the demo setups
Demo setup may need to scale horizontally. Determine if Docker or Ansible should be used for demo setups or if docker-compose is sufficient.
Research and document the process for horizontal scaling of the demo services
(Priority: Medium) Develop training materials so implementers can build reports https://openlmis.atlassian.net/browse/OLMIS-5256
Nifi Categories:
Configuring Controller Services for OpenLMIS
Processor configuration
Overview of the standard flow (Invoke HTTP, Evaluate JSON, Split, etc)
Nifi Registry interactions
Postgres database structure
The process for end-to-end automation - Document the dev workflow for reporting (like git flow documentation)
Getting JSI up to speed (requisition service is core compared to stock management)
(Priority: High | Jason) Deployment of NiFi to a production environment

TLS - Figure out access to the Nifi Registry behind HTTPS (Currently, HTTP is required without Sign-in and may be )
Backup all persistent NiFi Registry data
(Priority: Medium) Research improve loading passwords in Nifi Templates that were downloaded from the Nifi Registry
Research to identify if there is a central service that can store these passwords (Something like Ansible Vault)
Architect a way to store passwords for controller services and loading them into a specific template once it's been downloaded.
(Size: Large) Define an automated process for integration testing of the OpenLMIS v3 microservices
This needs to be fleshed out more this week
End-to-end
(Priority: High | Janette) Storing data in Kafka for replay https://openlmis.atlassian.net/browse/OLMIS-5258
Storing raw incoming data in Kafka so it can be replayed
Data for further processing comes from Kafka not API
Backing up Kafka data
Research options
Replicate to off-site
Copy raw files
Choose and implement one of the above
(Priority: High | Clay Starting on 22nd Aug in partnership with JSI) Evaluate all 12 reports and look for overlap between the reports.
@Craig Appl (Unlicensed) review this ->(Priority: High | Clay in partnership with JSI and VR) Evaluate each report to determine if data elements are present in OpenLMIS v3. If not available, we need to expose that and bring them back to the community as feature requests. https://openlmis.atlassian.net/browse/OLMIS-5265
Next Priorities:
Data Pumps
This is dependent on ingesting and standardizing the ingestion process to identify the pain points in the current ingestion process.
Suggested areas for Data Pumps
Stock On Hand to push stock events to Nifi or allow us to get access to all stock events through a single user instead of looping through each facility.
Pushing changes to the geographic hierarchy to the facility registry as updates
(Peter) Single Sign-on related epics
Review the OAuth process of OpenLMIS as a candidate system for Single Sign on
Role Based Access Control for Nifi Registry
Role Based Access Control for Superset
Identify actor roles for each component of the reporting stack. Who needs what access?
Compare that against other services like LDAP that can function to sign on to Superset

OpenLMIS: the global initiative for powerful LMIS software