Program Data Tech Research

Owned by Chongsun Ahn (Unlicensed)
Last updated: Apr 28, 2017 by Jake Watson (Deactivated)

The aim of this document is to keep notes on researching technology that will facilitate design, collection, storage and reporting of program data for OpenLMIS.

Form design and creation

The most important thing here seems to be is to use a form builder tool that can turn a form into the XForms format.

  • XForms - XML-based spec for forms
  • Another format, XLSForm which is easier to build forms with, since you can do it in Excel and there are tools to convert an XLSForm into XForm.
  • Tools that work with XForm (and XLSForm)
    • KoBoToolbox - can also import from XForm/XLSForm
    • ODK Build
    • Ona.io
    • Enketo

An alternative route is to build forms in DHIS2, which would not require XForms support, but the forms that can be created would not be as sophisticated?

Form data collection

Several tools that can facilitate this.

  • Enketo
    • Library for rendering forms based on XForms? Browser-based, offline-capable
    • Enketo Express - server for containing Enketo logic and some other stuff; good diagram here: https://enketo.org/develop/#libraries
    • Seems to just have APIs, would require significant effort to use it on its own, seems better to use it as part of a bundle (see KoBoToolbox)
  • KoBoToolbox
    • Packages Enketo in itself
    • Can export form into XForm or XLSForm
    • Can export form submissions into XLS, CSV, etc.
  • ODK Collect - Android only, get XForm from server, show form, can submit back to server or save onto device
  • DHIS2 - not sure of limit of what kinds of forms can be created?

Craig Appl has a really good comparison of similar tools here (for form creation, data collection and export): http://craigappl.github.io/Decisions-Decisions/

Reporting

No good/easy solutions here.

  • KoBoToolbox
    • Very basic data view and reporting; not good enough for our needs
  • DHIS2
    • More powerful, but doesn't really seem to have integration with XForms (although it appears many have tried, or have done custom integrations like scripts/ETLs)
    • Supports import of JSON and XML, but it has to match schema in DHIS2 (orgUnits, dataElements, etc.)
    • There seem to be some tools that may make it a bit easier (https://github.com/OpenFn/language-dhis2), but no finished solution


Running Notes

  • XForms - XML-based spec for forms
    • OpenRosa - Java reference implementation of XForms
    • Another format, XLSForm which is easier to build forms with, since you can do it in Excel (by ODK)
  • ODK
    • ODK Collect - Android only, get XForm from server, show form, can submit back to server or save onto device
    • ODK Aggregate - server to interact with Android client
  • Enketo
    • Library for rendering forms based on XForms?
    • Enketo Express - server for containing Enketo logic and some other stuff; good diagram here: https://enketo.org/develop/#libraries
    • Seems to just have APIs, would require significant effort to use it on its own, seems better to use it as part of a bundle (see KoBoToolbox)
  • KoBoToolbox
    • Packages Enketo in itself
    • Pretty decent form builder, can also import from XForm or XLSForm
    • Can export form into XForm or XLSForm
    • Can export form submissions into XLS, CSV, etc.
    • Very basic data view and reporting; not good enough for our needs
  • DHIS2
    • Doesn't really seem to have integration with XForms (although it appears many have tried, or have done custom integrations like scripts/ETLs)
    • Supports import of JSON and XML, but it has to match schema in DHIS2 (orgUnits, dataElements, etc.)

Craig Appl has a really good comparison of similar tools here (for form creation, data collection and export): http://craigappl.github.io/Decisions-Decisions/


Next steps

Research DHIS2 (ask DHIS2 contact and look into APIs):

  • Offline capability
    • From training, I do remember trainer said DHIS2 supports intermittent Internet access, but they do not recommend going fully offline to use the tool
  • APIs for form data completeness (find and put in APIs here) /api/<version>/dataValueSets
    • When collected (created and lastUpdated)
    • What it's for
    • Partially or completely collected (completeDate for completed, manual button click)
    • Who in DHIS2 collected it (storedBy)
    • dataValueSets for sets of valuesand /api/<version>/dataValues for individual values
  • RBAC, DHIS2 security model
    • OAuth2? Yes and HTTP Basic Authentication
    • SSO
  • Availability of form definition for export
    • /api/<version>/dataSets/<dataSetId>/dataValueSet to get a template for this data set

OpenLMIS: the global initiative for powerful LMIS software