2020-02-25 Sprint Transition
- Ben Leibert
- Mateusz Kwiatkowski
Date
Feb 25, 2020
Participants / Internal Stakeholders
@Felimone Amone Junior
@Matthew Kumbuyo (Deactivated)
@Chongsun Ahn (Unlicensed)
@Christine Lenihan
@Ben Leibert
@Sebastian Brudziński
@Aleksandra Ciesińska
@Mateusz Kwiatkowski
Retrospective
This was a good sprint during which we accomplished a lot a report-related work.
Showcase
@Mateusz Kwiatkowski focused on and/or showcased:
A number of tickets, many of which were moved to Dead, related to the removal of codes within the Superset reports.
@Aleksandra Ciesińska finished and/or showcased:
Planning
@Felimone Amone Junior plans on meeting with governmental stakeholders tomorrow, after which we’ll likely know the proper volumes for each Orderable, when quarterly requisitions should be considered tardy, what requisition forms should look like, etc.
We can build the "Warehouses Requisition Status" report such that it considers requisitions unsubmitted on or before the 25th of the final month of the trimestral period within which they’re due to be considered late. Tomorrow’s meeting with the ministry may yield guidance that we should use some other date. That’s not enough of a change, though, do delay building the report.
Because health facilities won’t be using stock-management, we’ll rely on their requisition forms to collect more data than we otherwise would. It would therefore be helpful if the forms used by health facilities can differ from those used by warehouses. OpenLMIS can already support this given that requisition forms can be customized by both program and facility-type.
@Felimone Amone Junior isn’t very concerned about our estimated hosting costs. He thinks it’s good that we estimated high and thinks it’s likely that our proposal will nevertheless be approved. Felimone thinks we should thus continue development as originally planned (with Superset reports, for example) but that we should have a plan-B as well.
It looks like Malawi’s server is sized reasonably. Over the past month, though, it looks as though the CPU on Angola’s main application server peaked at around 60 percent, and that it otherwise hovered around 10 percent. Angola’s hardware is quite possible over allocated (and ANGOLASUP-316 has been created to address this).
Batch Approval Implementation:
The work necessary to support Batch Approvals is substantial. (It was last estimated at about 6 weeks, and that was before yesterday’s design meeting may have somewhat increased the level of effort.) In addition to investing heavily in new business logic and UI functionality, yesterday’s design meeting left us with tentative plans to introduce substantial architectural changes to the backend as well. It’s unclear whether Mateusz and Aleksandra have the necessary bandwidth for all this.
Forgoing the planned ETL process might reduce our level of effort by about 20 - 30%. Doing so wouldn’t likely involve any throwaway work because we aren’t planning on any transformations. The tentative plan is simply to move data from the requisition service’s database into one owned by the new batch service - but not to transform it in any way. Although an anti-pattern, It would be viable to build the batch service such that it relies on the requisition service’s database. When time allows, we could then introduce an ETL (really an EL) process to move the data into the proper database. Updating the new batch service to use its own database should be pretty trivial.
Although direct use of the requisition service’s database would save us some processing time, the difference probably isn’t substantial. Because we’d be moving data rather than transforming it, we wouldn’t have to rely on NiFi. We’d mostly just need Kafka and Zookeeper. Even if budgetary constraints necessitated removal of the costly NiFi server, the datapump would therefore continue to work.
Q: Why are we tending toward a seemingly complicated solution build on Atom or Kafka rather than a simple pub/sub mechanism?
A: We needed a solution which wouldn’t involve modification to the core requisition service.
Because it’s unclear whether the current team has the bandwidth to introduce a data pump, and because @Chongsun Ahn (Unlicensed) already has familiarity with doing so, @Chongsun Ahn (Unlicensed) will setup Kafka to stream data form the requisition service to the new batch one. This will free @Mateusz Kwiatkowski and @Aleksandra Ciesińska to focus on the business logic and UI screens necessary for the Batch Approval feature.Neither https://openlmis.atlassian.net/browse/SELV3-86 nor https://openlmis.atlassian.net/browse/SELV3-115 will likely take long. It will take about another day to wrap up Angola’s support-reports because most of their charts will reside in AWS rather than Scalyr but don’t yet exist. We should aim to finish SELV3-86, SELV3-115 and Angola’s support-reports and then to focus on https://openlmis.atlassian.net/browse/SELV3-61 as exclusively as possible.
Action Items
Â