Quick update that I'm looking at Change Data Capture (with Debezium) as an approach to data pumps:
Still very much in exploration - no decision or course change yet. Though hope to get a sense for 3.6.
Streams changes from database, avoiding multiple writes (one to database, another as a published event)
CDC results in events in Kafka that look like the database, with schemas perhaps this is okay, however the event itself is often not the same higher-level semantic event we might think of (think of an Orderable vs. an Orderable with a Program Orderable).
One potential solution is to have a stream processor, perhaps even the service, which is responsible for making more semantic events for downstream clients.
Begs the question what, if anything, we'd use Nifi for.
(for next time)
DHIS2 followup on how (keeping this on the agenda to discuss the plan prior to implementation, however implementation isn't pressing on this just yet)