2018-10-30 TC Meeting notes

Owned by Josh Zamor
Last updated: Nov 13, 2018
3 min read

Date

7am PST / 4pm CET

https://zoom.us/j/737202874

Attendees


Discussion items

TimeItemWhoNotes
5mAgenda and action item reviewJosh
15m

Followup on audit log initializer

  • What other snags are making development work painful?
    • Memory pressure / capability of development machines?
    • Multiple docker-compose files?
    • Long compose commands?
    • Lack of hot-reloading?
Josh Zamor
  • We completed  OLMIS-3078 - Getting issue details... STATUS which will help production data sets, however for demo data we would still iterate over all the demo data records on the first start as they don't have snapshots yet.  Is a slow startup still / more an issue?  With the increased size of the demo data that we're adding, and thoughts to have millions of records, do we think we have an issue?
  • When Ref Distro is started, DB usage is sustaining 100% CPU for > 5 minutes
10mOrderable versioningChongsun Ahn (Unlicensed)
  • Continue the show and tell on the how
(for next time)Data Pumps: DebeziumJosh Zamor
  • Quick update that I'm looking at Change Data Capture (with Debezium) as an approach to data pumps:
    • Still very much in exploration - no decision or course change yet.  Though hope to get a sense for 3.6.
    • Streams changes from database, avoiding multiple writes (one to database, another as a published event)
    • CDC results in events in Kafka that look like the database, with schemas perhaps this is okay, however the event itself is often not the same higher-level semantic event we might think of (think of an Orderable vs. an Orderable with a Program Orderable).
      • One potential solution is to have a stream processor, perhaps even the service, which is responsible for making more semantic events for downstream clients.
      • Begs the question what, if anything, we'd use Nifi for.
(for next time)DHIS2 followup on how (keeping this on the agenda to discuss the plan prior to implementation, however implementation isn't pressing on this just yet)

Josh Zamor

Notes


Do we want to change time (DST)?

  • It sounds like not (if there is someone in a timezone where this becomes harder to attend because of daylight savings changes, please let us know and we'll try to accommodate)


Audit log initializer

  • Currently bogged down in CCE alerts under demo data (the developer workflow primarily)
  • Stock Mgmt screen was slower - not sure if this is from # of new orderables, or start up sequence of audit log initializer
  • Are we starting ref distro locally?
    • Some say yes
    • Only if changes in several services, otherwise push and see in test.  Run ref distro to test demo data.
    • Run ref distro migration changes (working on Jenkins migration test problems) - running it locally.
    • In UI dev you don't need to run ref distro, the UI dev can work off a deployed cloud instance
    • Some / many aren't starting ref distro locally
    • Would be nice to catch build failures before they got pushed (and so would make sense if people weren't starting it locally)
    • Biggest problems:
      • time consuming to launch ref distro
      • configurations (after some months of not running, the changes to setup required debugging / migration)
      • Changing a single module (service?) - we don't have documentation to show how that's done.
    • Negative for marketing to new devs / users:
      • Native developer experience
        • e.g. IntelliJ idea running tests (would need to disable the login/auth, which might accidentally get committed, for integration tests)
        • Keep development instance more up to date so that we could build outside of docker (it's more efficient)
        • NodeJS issue with version 6 and version 9 - dev-ui is version 7, on mac only 6 and 9 are installable.
      • Complicated setup - settings.env and the profile that's used.
        • When services haven't all started on login, you'll get an error.
      • Big machine - Make services more efficient, explore service meshes (though would require big internet), etc


Orderable


  • Thread:  https://forum.openlmis.org/t/orderable-edit-design/455/16
  • No longer extends base entity due to compound primary key
  • Foreign keys from other tables sometimes needs to also have a version indicator
  • Main lessons: 
    • Our tools really don't like this:  Hibernate and the ORM impedance mismatch
    • Composite keys are difficult, if we were to do this again, try to stay away from composite keys (there were a couple other potential solutions but couldn't get those to work)

Action Items

  • Josh Zamor to make dev forum post on pain points (and do this again on next TC - timebox but keep it going)

OpenLMIS: the global initiative for powerful LMIS software