Team Ona Release 3.5 Backlog Grooming Brainstorming

Owned by Craig Appl (Unlicensed)
Last updated: Aug 22, 2018
4 min read

This brainstorming was originally completed in Aug 2018

End state for the 3.5 Release Reporting Stack (unordered):

  • All Reports in the Gap Roadmap have been scoped and mapped against the OpenLMIS v3 APIs
    • This includes evaluating whether OpenLMIS v3 has the features available to generate each report
    • A list of reports that can't be created due to feature discrepancy has been published to manage expectations
  • A list of candidate endpoints for data pumps has been created to inform the development of those features.
  • Multiple teams have experience with and are actively developing reports in the reporting stack
  • Automation of the end-to-end reporting stack has improved minimizing the amount of human intervention required
  • CI/CD processes for the reporting stack have been developed, documented and tested for at least 1 report
  • The Nifi Registry supports TLS and is actively being updated by developers as the flows are developed
  • An evaluation has been made for the Single Sign-on needs and capabilities and a go/no-go decision has been made

Epics:

We want to move to smaller sized deliverables by creating an Epic that can be achieved in one or two sprints. We would like to get to a point where the backlog grooming is focused on scoping an Epic. For example, these N epics together will get us to the bigger goal of the 3.5 release.

  • (Priority: Medium) Improve Automation of loading Superset  OLMIS-5270 - Getting issue details... STATUS  
    • OLMIS-5285 - Getting issue details... STATUS
    •   OLMIS-5286 - Getting issue details... STATUS
    • When you create a chart in Superset, each chart has a different configuration. If it's a bar graph, you need to know the configuration that's specific for Bar Charts in the JSON that we define in Nifi. We can solve this by adding JSON in Nifi per chart type.
    • Auto loading the data source, we have to create the data source in Superset and then create the .yaml file
  • (Priority: High | Antonate) Ingesting referencedata microservice and store in OpenLMIS v3 postgres data structure OLMIS-5266 - Getting issue details... STATUS
  • (Priority: High | Clay) Perform the following for the Reporting Rate Report
    • (template) Mapping from each report to data table required for that report, and from that to  the API endpoints storing that data
      • For each report we want to build
        • A list of data needed for that report (the columns)
        • A list of API endpoints you need to access to fetch that data
      • Use this to determine
        • If there are any reports we do not have all the data for
        • If there are ways to represent multiple reports via less than a single table per report
  • (Priority: High| Jason)Reporting Jenkins  OLMIS-5257 - Getting issue details... STATUS
    • Standing up Jenkins jobs for the demo setups
    • Demo setup may need to scale horizontally. Determine if Docker or Ansible should be used for demo setups or if docker-compose is sufficient.
    • Research and document the process for horizontal scaling of the demo services
  • (Priority: Medium) Develop training materials so implementers can build reports  OLMIS-5256 - Getting issue details... STATUS
    • Nifi Categories:
      • Configuring Controller Services for OpenLMIS
      • Processor configuration
      • Overview of the standard flow (Invoke HTTP, Evaluate JSON, Split, etc)
      • Nifi Registry interactions
      • Postgres database structure
    • The process for end-to-end automation - Document the dev workflow for reporting (like git flow documentation)
    • Getting JSI up to speed (requisition service is core compared to stock management)
  • (Priority: High | Jason) Deployment of NiFi to a production environment
    • TLS - Figure out access to the Nifi Registry behind HTTPS (Currently, HTTP is required without Sign-in and may be )
    • Backup all persistent NiFi Registry data
  • (Priority: Medium) Research improve loading passwords in Nifi Templates that were downloaded from the Nifi Registry
    • Research to identify if there is a central service that can store these passwords (Something like Ansible Vault)
    • Architect a way to store passwords for controller services and loading them into a specific template once it's been downloaded.
  • (Size: Large) Define an automated process for integration testing of the OpenLMIS v3 microservices
    • This needs to be fleshed out more this week
    • End-to-end
  • (Priority: High | Janette) Storing data in Kafka for replay  OLMIS-5258 - Getting issue details... STATUS
    • Storing raw incoming data in Kafka so it can be replayed
    • Data for further processing comes from Kafka not API
  • Backing up Kafka data
    • Research options
      • Replicate to off-site
      • Copy raw files
    • Choose and implement one of the above
  • (Priority: High | Clay Starting on 22nd Aug in partnership with JSI) Evaluate all 12 reports and look for overlap between the reports.
    • Stock On Hand Report  OLMIS-5259 - Getting issue details... STATUS
  • Craig Appl (Unlicensed) review this ->(Priority: High | Clay in partnership with JSI and VR) Evaluate each report to determine if data elements are present in OpenLMIS v3. If not available, we need to expose that and bring them back to the community as feature requests.  OLMIS-5265 - Getting issue details... STATUS
  • Next Priorities:
    • Data Pumps
      • This is dependent on ingesting and standardizing the ingestion process to identify the pain points in the current ingestion process.
      • Suggested areas for Data Pumps
        • Stock On Hand to push stock events to Nifi or allow us to get access to all stock events through a single user instead of looping through each facility.
        • Pushing changes to the geographic hierarchy to the facility registry as updates
    • (Peter) Single Sign-on related epics
      • Review the OAuth process of OpenLMIS as a candidate system for Single Sign on
      • Role Based Access Control for Nifi Registry
      • Role Based Access Control for Superset
      • Identify actor roles for each component of the reporting stack. Who needs what access?
      • Compare that against other services like LDAP that can function to sign on to Superset

OpenLMIS: the global initiative for powerful LMIS software