Requisition Batch Approve service design meeting (Kafka + Debizium)
Attendees
@Mateusz Kwiatkowski
@Klaudia Pałkowska (Deactivated)
@Aleksandra Ciesińska
@Chongsun Ahn (Unlicensed)
@Wesley Brown
@Sebastian Brudziński
@Josh Zamor
@Ben Leibert
Notes
Project Casper - streaming architecture to sync OLMIS v1/v2 with OLMIS v3
3 parts:
source system (requisition db)
Kafka
target system (new Requisition Batch service)
https://casper-elmis.a.openlmis.org
Debizium is required to be installed on requisition db
topics in Kafka
Kafka connect - a wrapper around Kafka for defining connectors, config has a connection to the source database, whitelist contains all tables that we want to watch, those would correspond to topics in Kafka
Kafka and Kafka connect are established for reporting stack so we can use its docker-compose as an example
feed vs debizium
would be nice to include 3 services (kafka, kafka connector and zookeeper) into the OpenLMIS docker-compose file
it would be nice to have a service that data would be streamed into and could be used for demo purposes
Next steps
when we could move those services into core
write tickets
add kafka and others to OpenLMIS Ref Distro.
start building new service.
stream the changes out of Requisition service.
stream data into batch service (aggregate on write).
remove kafka and others from reporting stack, as it's now in ref distro.
https://gitlab.com/openlmis/casper-pipeline
https://gitlab.com/openlmis/casper-pipeline/-/tree/master
GHSC-TZ Migration (Project Casper)
OpenLMIS: the global initiative for powerful LMIS software