Open questions:
Notes
- Reporting stack process groups
- Reference data connector
- Requisition connector
- Materialized views - refreshes the first two connectors, basis for indicator queries
- Waits three minutes (hardcoded setting) on startup
- Postgres db schema is created in init db, not in Nifi
Using the Nifi UI
- Currently most of the db connection pool settings are hardcoded (except the db password which is in the .env file), but there is a ticket to automate settings these from the env file
- Right click within process group > variables > specify your list of variables for later reference using NiFi language
- We should write a ticket to make the user-client property configurable
- Nifi uses user-based token (using administrator user); we'll need to design intended use for service-based token and service account/API key tokens; reporting stack likely should use API key tokens OLMIS-3861 - Getting issue details... STATUS
- Each OpenLMIS API endpoint is converted to table using joltspec
- Antonate Maritim (Unlicensed) will write a a ticket for how to handle paginated responses from the API. Note this ticket may not be necessary to take on as Mateusz Kwiatkowski mentioned working on something to have page size = ~2B
Requisitions connector
- UpdateAttribute modified date to beginning of Unix epoch to get all requisitions on startup
- Runs every day at midnight
- cronmaker.com to generate cron expressions
- Trigger uses modified date of yesterday
- Requisition jolt transform splits requisitions and their line items into four tables
NiFi versioning
- Shows registry clients
- Right click on a connector, version, show local changes