Contents

Global Digital Health Forum 2018

When: December 10 - 11th 2018

Where: Washington DC

Attendees: Rebecca Alban (Unlicensed), Vidya Sampath, Swetha Srinath (Unlicensed) and Mary Jo Kochendorfer (Deactivated)

The following is a joint trip report. Please note that the session notes are quite detailed and some are rough notes. Feel free to leave questions and comments for the authors.

Overview

Team spoke on two panels (Mary Jo and Vidya), presented two posters (Swetha and Rebecca), attended lots of sessions, and crammed in lots of meetings in two very full days. Some photos and table of Contents with LINKS below!

Rebecca presented a poster on OpenLMIS.

Agenda

Here

Follow up

Follow up for OpenLMIS Stewards

BAO: Rebecca to send Trusted Partner information to Steffen and BAO systems. - DONE
Zambia: Mary Jo to bring up SMSforLife to JSI team and understand what their thoughts are of the application and potential use of it or integration with it in Zambia. Rebecca to follow up and see if the JSI team would be willing to share more details on the online training developed with the community. A potential addition to the OpenLMIS Implementer Guide.
Bahmi: Follow up with Steven (swanyee@intellisoftkenya.com) about an opportunity with MedSource. - DONE
Jembi: Mary Jo Kochendorfer (Deactivated) to follow up with Chris about ways we could potentially work together. Mary Jo met will him after our panel and will continue the conversation in 2019.
Mezzanine: Mary Jo Kochendorfer (Deactivated) to send the Zambia RFI. - DONE
Mezzanine: Mary Jo Kochendorfer (Deactivated) to ask for more information and the source code is on SMSForLife.
TZ/Alpha: Mary Jo to follow up and obtain the diagram he created for the eHealth architecture for Tanzania. - DONE
Miguel Sitjar (Palladium): Rebecca to send follow up email providing him with community resources and contact information for SolDevelo that he and his colleagues can use for potential implementation Guatemala- DONE
Meaghan WHO: Rebecca to send the follow up email CCin Vidya and Mary Jo. We will connect about Digital Health Guidelines and Ask for the contact information for the regional ICT point people Derrick, Mr. Ba, and Mark. - DONE
I-TECH: Mary Jo Kochendorfer (Deactivated) to follow up on the Notice C efforts and connecting with the OpenELIS product. She will reach out to Jan Flowers, Casey and Joanna. The group plans to meet in the new year. https://proposals.digitalsquare.io/91
Shifo Foundation: Mary Jo Kochendorfer (Deactivated) met Nargis Rahimi and will follow up to learn more about SmartPaper to see if it could be useful for the OpenLMIS implementations. Brandon has also been connected with Nargis. Plan is to schedule a demo in the new year. Potentially very exciting.
Abi Gleek (Every1Mobile) Rebecca Alban (Unlicensed) to connect with Abi Gleek, who implements the Naijacare app (see session notes below) to learn more about their digital ordering system and e-learning modules that they have built into Naijacare

Presentations

H ere

Session Notes

Facilitating Collaboration to Accelerate Scale and Improve Digital Health Global Goods

Attendee: Rebecca and Vidya

Presenter (panel): Mary Jo Kochendorfer (Deactivated), Matt Berg (Unlicensed), Amanda BenDor (Unlicensed), Chris Seebregts (Jembi), Steven Macharia (Bahmni)

Description: Digital Square, an innovative co-investment global program led by PATH aims to serve as a convener in the digital health community, bringing together practitioners to share their work, lessons learned, and future needs. This helps organizations to work together to implement digital health tools that are adaptable to different countries and contexts, also known as Global Goods. To foster transparency, collaboration and synergy, Digital Square uses an open application platform including community feedback to align investments in digital health systems. The open application process provides a unique opportunity for practitioners to learn key details about, and comment on, each other’s tools and implementation plans. This session will describe the innovative open application process and feature global goods awarded through Digital Square including the OpenLMIS, Community of Practice, OpenCRVS, OpenSRP, and Bahmni.

Digital Square- lives at PATH but is a consortium of donors. They are a funding mechanism, and coordinate digital health investments

Open Proposal process- community feedback from peer review committee and governing board, comments, and create coalitions. Transparent process

Bahmni-open source Hospital Information System (HIS) and EMR; does not require custom software development; about 50 implementation

Jembi- OpenCRVS (civil registration and vital statistics); register children, deaths, big unmet need; works with FHIR 7, HopenHIE, and DHIS2. Supports OpenHIE architecture; can interoperate with low demand service like civil registry, with a low demand service like vaccination

OpenSRP- focus in care at facility and community level. Integrates with national health systems like DHIS2. Does Client ID, client management. OpenSRP community Canopy reporting stack is result of DigitalSquare investment

?What are challenges with Digital Square open proposal process?

Benefits- standards are valued, collaboration with other open source is encouraged. Peer review was helpful. Digital Square has been helpful for some in terms of opening up other funding too. Organizations appreciate the validation as a ‘global good’.
Challenges- it is a new process; smaller communities might have trouble keeping up with the administrative lift. Sometimes the amount of money is now known, so that can cause confusion and is hard to plan for. Some organizations don’t necessarily want the proposal process to be so open (don’t want to share their idea).
Can be a lot of work for not a lot of money
Amanda’s role is to ensure process is clear, easy for participants. Have to divvy up the pot of funds to spread across multiple global goods
5 million dollars invested in 27 global goods initially, went up to 12 million
Global health funding in general is not directed toward developing core technology. This funding model is better than giving directly to the implementers to decide what to do with it . Longer-term funding can be a gap
How to harmonize what you are doing at the program level, and balance it with global goods- make it easier for people to build into the health eco system
Fragmented donors and fragmented funding can make it hard to develop global goods, Digital Square and DIAL are easing that situation
There is no business model to sustain things -like having a product manager. Its important to think about blocking off specific capacity
Could think about building in some mechanisms for countries/implementations to pay a licensing fee of some kind. This could encourage sustainability to fund ongoing maintenance of the global good. Software is a ‘leaky boat’ that always needs care
Suggested improvements for proposal process: support for intellectual property and general community building, also the other ‘soft’ thing around OpenSource. People tend to volunteer for actual coding, other things such as management, not so much

Suggested investment in Dev ops-
Connect-a-thon and other capacity building to teach about FHIR and standards

Impact of electronic systems on health service delivery in Zambia

Presenters: Wendy Bomett (JSI Zambia), Chris Opit (Unlicensed) (JSI Zambia)

Attendee: Mary Jo Kochendorfer

Provided highlights of the implementation of eLMIS and Facility edition.
Wendy went over the business processes.

Receiving
Inventory management
Dispensing

Chris will talk about the impact of an electronic systems. Emphasized the importance of having manual systems in place prior to rolling out the electronic system.

Automated reports generated from electronic systems
Reduction in workload at SDP and Central level (no specific figures showed)
Improved efficiency and accountability (due to increased in reporting rates)
Commodity availability - before the system they can see what facilities have overstocked items to “share” while awaiting their consignment
Timeliness (78% fully automated 66% that are not fully automated)

Workload decentralization has helped with data entry workload

District facilities still have the most workload but central is down to zero.

Improved Data use

Tracked of number of sessions (not sure if they mean login sessions)

Improved report timeliness by targeted training and championing. Having a system itself can only improve so much. Still needs training and human prioritization.
Reduced wastage: eLMIS cuts expired drug inventory in hospitals.

Uses the system each monday to check the stock status to know if they have overstocked items and can redistribute. Showed a graph of reduction in ARVs (from 1.5% - 0.2%)

DEMO
Select the product source, program area, Dispatch Number (from the dispatch Note)
Select each product and batch information
Allow for entering in their own batch numbers
Then shows the stock control card which can be printed.
Select date, program and receiving node, person
Issue voucher is generated for printing
Select program area
Have to add each product one by one
Quantities and remarks.
Client ID, Gender, DOB
Seemed to only be for ARV clients
Goal is to dispense in under 1 minute
They enter the number of days needed for the medication. The system says how many bottles.
Subsequently, the daily registry is updated.

Facility Edition - the web version was showcased.
Received products
Issue to the dispensary
Adjustments
Physical Count
Dispense (doing a bulk upload from the EHR)

Discussion:

Zambia is looking to automate down to the community level.

What electronic systems do you use for supply chain management in your country?
What are your pain points?

Questions:
Started work on that. Collaboration is very difficult. Interface is working in two - three more facilities. Now are testing the web-enabled version of facility edition.
Chris: yes… but didn’t clarify and mentioned paying attention to security
Wendy: yes. Needs to rolled out by the government. No report no product principle helps move to systems.
Data quality and completeness. There are data validations built into the system so it is harder to “quickly submit” anything. You have to true up things.
Chris: big issue. When we developed the training program, we looked at sustainability. How to continue without us re-training? We train people on the job (hands-on training). If you go to the facility, you can train everyone in the facility. Facility then takes over in training interns and new people. District health information officers will also train facilities.
Wendy: there is an e-learning module for them to print out a certificate. Annual meetings to discuss the data and will hold a training on the 3rd day or something.
Wendy: we have a report who can look at the adjustments which HFs are making. Particularly with facilities that are automated the entire way.
LAN FE system
They wouldn’t abandon because built on manual processes
Sustainability plan for the ministry to own.
Everyone wants something to track transactions. Phones won’t work because everyone has at least 100 products.

Have you automated pharmacy dispensation?
Are you interoperable with SmartCare? (from CDC - Dan Rossen)
Are you following OpenHIE principles?
Did we face transparency challenges?
Challenges with uptake of the system at the facility level?
Sometimes HF report on time but then they have to go back and resubmit?
Who owns the data?
MSH Question: you many facilities … how do you manage staff turnover/attrition and training?
WHO Humphrey Question: Issue with leakages (paper and what is on the ground discrepancies). Are there issues with redistribution (which allows for more opportunities for leakages).
WHO Question: What about internet access?
If JSI was to walk out today, would the system still be used?
Question: For low-throughput clinics, have you thought about using mobile?

A system should be there to help a ministry not lose money

Opening Plenary- How funders are operationalizing digital health principles

Attendees: Rebecca, Swetha, Mary Jo

Emphasis from CDC and USAID on the importance of them shifting from supporting bespoke tools to self-reliant tools and Global Goods
GIZ mentioned Digital Square and DIAL as platforms that they would be interested in leveraging
In order for donors to follow these principles, countries actually need to have a digital health strategy. So there is a call to countries to create these plans in order to get funding in this area

Strengthening primary health care through digital tools

Attendee: Swetha Srinath (Unlicensed)

Speakers: Maryanne Mureithi; Jamil Zaman (Bangladesh), Ephrem Lemango

Moderator: Uju Aderemi

Medic Mobile [Ethiopia]

Focus of intervention:

Care coordination / referral network

Overview:

Community health innovation network – partnership with living goods
Integrate community activities – health facility care
Increasing reach through technology

Why Referral

High priority cases
Time-sensitive
Closing the loop

Referral pain points

Lack of referral tools / loss of tools [paper systems – supply chain]
Accuracy of reporting
Px / Patient experience [long lines]

MVP Design

Digitized referral systems
Referrals trigger tasks
Referral targets
Facilitating communication

Challenges

Internet communication issue
Performance of app
Keeping the workers engaged
Involved them in design process

Mobile Job Aid for H.E.Ws [Ethiopia]

Focus of ppt:

RMNCH

Overview

Health Extension Program
Intro: HEP is one of the strategies adopted by the government of Ethiopia to achieve universal coverage of primary health care among rural population
Goal: to create a healthy society and to reduce maternal and child morbidity and mortality rates.
Overview: Program that is deeply rooted in communities, providing primary level preventive activities to household members. In addition to community activities, HEP also provides health post–based basic services, including preventive health services such as immunizations and injectable contraceptives, and limited basic curative services such as first aid and treatment of malaria, intestinal parasites, and other ailments. Case referral to health centers (HCs) is also provided when more complicated care is needed.

Why Mobile?

Coverage / equitable coverage
Timeliness of visits / care

Components of tool

Automated reminders and follow up
Mobile job aids
Leading to improved referral work flow

App points of intervention

HEW interaction with client: ANC 1 – ANC 4
ANC 1 – sent to health center for first one, then sent back to community
2nd, 3rd, 4th sent to community / HEW
After delivery, HEW notified to provide post-natal care

HEW Application

Provides info on location where visit should take place
Provides timeline for timeliness of visit
Provides referral management etc.

Status / Progress

Issues: Capture at health facility level when delivery happens – needs to improve
Political commitment: Needs to start from minister level
Risk: Intermittent inaccessibility / increasing technical complexity / changing and expanding vision and scope

eMIS (Save the Children Bangladesh)

Focus of intervention:

eMIS

Partnership:

MOHFW

Problem
Included supervision, though cumbersome: Paper-based system included a supervision process to ensure that FWAs and HAs were visiting the households and correctly completing the forms, but it was cumbersome to
Supervisor role: Supervisors required to visit the households or satellite clinic that the HAs and FWAs supported to verify the data collected and to review the registers and reports.
Lack of supervisor feedback: Under the paper system, FWAs and HAs did not receive direct feedback on their performance in completing the required forms
Data use not regular: Under the paper-based system, FWAs, HAs, and supervisors did not regularly use the data they collected to improve quality and coverage of services. The information collected was primarily reported to the higher level,
Only option for referral: The only way to know whether a client completed a referral was through self-reporting during follow-up visits, which could be occur between two and four months after the initial visit. Facilities to which the clients were referred did not necessarily keep records of referrals, and the reporting mechanisms for referrals were weak.

Inability to track patient along continuum of care
Reporting delay / slow and error prone
Lack of sync
Paper-based system

Digital Application – eMIS

Registers needed to be unified / simplified (across continuum of care)
Indicators (for appropriate stage) included

Digitization process
Client unique identity – pregnancy registration
Provides prompts and alerts based on protocol
Protocol embedded into the tool
Longitudinal health records
Stratify clients by risk type
Schedule work plan for providers
Monitor performance of healthcare provider
Assess health facilities
Manage health commodity inventory
Data Services
Non routine data collection
Automated analysis to generate new information
Map location of facilities
Reduced workload of HAs and FWAs: eMIS has significantly reduced the workload of HAs and FWAs and errors that were related to transcribing the data to different paper forms.
Time spent compiling info: When using the paper-based system, FWAs and HAs would spend one to two days each month compiling the monthly HIS report. With the eMIS, they can press a button and the report is generated instantly.
Better tracking of patients: Unique health identification system (master client index) established in the eMIS through a population registration system has improved quality of care by allowing health workers to track individuals over time and ensure the continuum of care At the facility level, service providers can pull up an individual’s information by using the unique health ID and retrieve all stored data from the population and service databases. FWAs and HAs can also retrieve services provided to their clients as well as check on referrals using their tablets with the unique health ID.
Service mapping: Dashboards generated from the data collected on the tablets have helped district and national-level managers plan services to ensure the continuum of care. A manager can pull up services by a particular provider to view data by date. This reduces the need for supervisors to travel to the field to check all aspects of quality of care to determine where improvements are needed.

1. Provider System
2. Health System Managers
Results with eMIS

Questions

Was paper replaced in Bangladesh?

Double reporting – in initial stages; when reliance on digital increased, paper was replaced in Bangladesh
2-3 months lead time: for HCPs to get used to smart phones

Are these tablets in Bangladesh / Ethiopia – real time offline?

Ethiopia: moving towards tablets, which should work offline
Bangladesh: All data is continuously synching – when internet is available, synch happens in background
OpenSRP for case management – how to make sure you refer without duplication
CHWs refer patients within app when activities initiated, they have to go through app to trigger referrals all activities are coordinated using the platform, and CHW has to be registered

Data Privacy?

If we want to check if follow-up is done, we only display CHW performance, as opposed to patient status / data
Bangladesh: govt doesn’t have legal framework RE: data privacy – working with govt based on paper based privacy policy – a) Also patient identity based on number, not patient name pre-referral and b) data only becomes available to next level provider IF patient is referred.

Linkage to EMR

CHWs refer patients within app when activities initiated, they have to go through app to trigger referrals all activities are coordinated using the platform, and CHW has to be registered
CHW Registering – at health facility + community?
CHWs – “hashing”? non deterministic

eReferral – patient level challenges – any cases of where patients referred but they didn’t show up?

Digital solutions are not the only solutions
This platform encourages continued community worker – persistence – ensures patients get care

WHO's 1st ever Guidelines for Digital Health Interventions for HSS

Attendee: Vidya

Panelists: Garret Mehl, Tigest Tamrat, and Smisha Agarwal

Notes:

PATH, as the support org to WHO, convened a session with the WHO team that has put together the very first WHO Guidelines on Digital Health Interventions for HSS.

Team consists of Garret Mehl, Tigest Tamrat, and Maeghan Orton (latter two involved with early CCPF work)

The guidelines for formally approved by WHO board this week and will be made publically available early next year. PATH is helping put together an interactive website to accompany the print publication. The guidelines will also have helpful language/visuals to show how these interventions map to Universal Health Coverage (UHC) goals.

This work is part of the larger Digital Health Atlas compendium that this team is also leading (a WHO global technology registry platform).

The team first developed "Classification of Digital Health Interventions" and the Guidelines map to this classification.

The third deliverable is a "Planning and Costing Guide for Digital Health Interventions" which is set for release next Spring.

Monitoring and Evaluation of existing digital health interventions:

The team highlighted the usefulness of the mERA checklist which has been around for a couple of years now. For impact evaluations, they recommend the following study designs:

Controlled before and after;

Stepped wedge RCT;

Interrupted time series studies

More information on mERA here:

https://www.bmj.com/content/352/bmj.i1174 (paper summary)

https://www.researchgate.net/profile/Amnesty_Lefevre/publication/311738723_Monitoring_and_evaluating_digital_health_interventions_a_practical_guide_to_conducting_research_and_assessment/links/5858cdf908ae64cb3d47f7d3/Monitoring-and-evaluating-digital-health-interventions-a-practical-guide-to-conducting-research-and-assessment.pdf (detailed assessment guide including the checklist)

Takeaways:

OpenLMIS should start folding in language about both principles of digital development as well as the Digital Health Interventions classification and guidelines in our future proposals and when talking about our current work.
Tie efforts to UHC when possible
OpenLMIS team (see follow-up/action section) is in touch with the WHO team about how to include the Implementation Toolkit in the interactive website that PATH is working on.
Reference the mERA assessment guide for future digital health work

Putting the Patient in Charge- new tools to help clients manage their own health data

Attendee: Rebecca

Online self-administered screening tool for improving TB detection among students in Ethiopia (MSH)

In Ethiopia, they have case-finding issues. ⅓ cases are undiagnosed. Targeting universities. Students were active partners in the initiative , made suggestions on how to implement, draft the self-screening checklist, and how do outreach, etc.

Online screening to be the first layer of assessment (rather than needing a HCW to examine each individual)

The tool linked with CommCare for those whose screening showed suspected TB
Targeted students on a free wifi system that they were already using (good use of resources); did targeted outreach in free wifi hotspots

Maternal Health in Liberia(D-Tree)

Want flexibility to attend different health facilities throughout pregnancy/delivery
About 80% of facilities in Monrovia are private. How do you adopt across a lot of actors; public and private. This can act as a central repository of data that bridges the public/private gap

Tuberculosis in Thailand (D-Tree)

Migrant populations tend to lose touch with health system and default on

-->NFC token stores the client’s information. It can be a card, sticker, bracelet, etc. Client can keep that card and bring it with them. Its low cost (10x less expensive than biometrics).

If the card is lost, the data is encrypted and card can be replaced. It is language agnostic
Data is kspe on the phone and when there is connectivity it is synced with a dashboard

DOT Fertility App- to help manage fertility (Cycle Technologies)

70% of the world will own a cell phone by 2020

DOT fertility app uses user info and machine learning to track her cycles; help prevent or plan pregnancy

Ways to think about our own technology (OpenLMIS)

Context- what problem you want to address. What is currently being used
What are the reasons for non-use? (cost? access?)
Usability- does it ‘delight’ the user?
Evidence- what impact does research show? What usability research to you have to show users understand and like it?
Scale up/Sustainability- what marketing is needed? What is your audience willing to pay? Are there business models (upgrade models, subscriptions, etc.) to consider?

Challenges of Innovation at Scale

Presenters: Wayan Vota (Digital Health Director, IntraHealth International) Jonathan Jackson (Dimagi), Dr. Iniobong Ekong, Clayton (Dimagi), Carmen Sant

Attendee: Mary Jo

Dimagi, Jonathan Jackson - the Digital health principles work for scoping (not at scale). Scope and scale are very different. Scope/pilot is what you get done on a certain budget for a certain set of users. Scale is really hard and procurement, training, data use, etc.

We didn’t have the initial challenging conversations at the beginning. We don’t have good frame-of-reference for those conversations.

For example, 130,000 scale was slowed due to procurement of hardware. Gave time for the conversation to happen because scope was deployed for 18 months before scaling.

Dr Iniobong Ekong - Nigeria (FCT eHealth Project).

ICT + Health systems = universal access.

Bought notebooks for doctors to support doctors in servicing rural. Access increased by 247%. All things going great.

Big challenge, had to still complete the manual forms and electronic systems in parallel.

Need to make sure the ART reporting is integrated into the EMR. Clear requirements and standards need to be in place. Now now new EMR can be rolled out without a clear ART module meeting the standards.

Top down doesn’t work. Standards are needed. Continuous evaluation of systems.

Clayton CTO - Talked about the Bihar scale up

130,000 CHW using CommCare.

5 years of working on the app before we scale.

Goal was to have non technical teams design and build the app.

242 servers to host the system right now.

What they had to focus on from 2017 was about tools for deployment. Not the actually application.

Carmen Sant (Research Assistant)

Health systems perspective. SMS for Life. 2009 over 5000 health facilities in TZ. Monitoring the stock outs. Ran for 4 years collecting the weekly data and then it was cancelled.

We thought it was really good from a technical perspective. Focusing on anti-malaria.

Surveillance without a response will not bring you to where you want to go. There wasn’t a plan for how to respond to the ‘stock outs’. Who was responsible for the stock outs?

Perhaps the program didn’t involve all the stakeholders who needed to be there to realize this gap.

Pilot tech was inexpensive.

Costs at scale supposed a large burden for the government was quite large.

She overlayed the timeline from both the SMS for life and the government efforts. There were parallel efforts and the SMS for life app didn’t change or wasn’t considered.

Focus of visibility, wasn’t enough. What about the action? A costing exercise would have been useful. Agile technology, program design, and project management would have been useful to allow for pivots.

Key Takeaways and Discussion

In a pilot a system may “not be in scope”, but once you scale it will come into play. You don’t want to fill out two systems.
How do you think the donor pression on national e-health architecture will play into things?

Jonathan - is excited about the focus but it will be very challenging for companies reliant on project based funding. We will need to change how outcomes work since folks will not be excited to work on things which are not in your best interest.
Dr Iniobong Ekong - there was a uforia around technology so folks were resistant of policy and regulations since it could slow the excitement of rolling out technology. Need to make policy useful and helpful. Making it easier to roll out things with clear standards.

CRS Marcus: How do we change the culture and begin to learn from our mistakes? We are worried that if we are not successful we may not get funding again.

Clayton: There is still a root to failure. We sometimes continue to not allow things really fail fully.
Jonathon: We gave ourselves a break for structural failure. We aren’t good at de-coupling failure. If no one is a owner to make decision, then you fail. Governance issues will force failure. When we take a project with no funding after six months. We know it will fail.

Waylon: Brought twitter to Cuba. Only had a specific amount of money for the cost. Pulled it. Thoughts on how to do that better. Is it a set up failure or what?

Carmen: Both paying people to use the SMS in the initial roll out. The government wasn’t willing to pick it up. Also, the team wasn’t really learning as they go. RDT were added but no one new that they could use it now. Need constant training. Why are we using “systems thinking”? Perhaps it is complex and we aren’t open to that.

Question: Scope versus Scale. Sales mode you hide the complexity and then Scope phase you highlight the complexity. What is the honest broker?

Jonathon: within scale (there is a huge amount scope in scale - like data security). That has to be driven by honest brokers around security Tech companies cannot ask for that given that they will be view. Let’s name bias. We are biased, doesn’t that mean your point isn’t valid. Can you create the right structures to answer the questions? Consultants should be determining if they should still be useful. Technology firms should choose the technology.
Dr Iniobong Ekong - allow for learning.
Clayton - you should account for market prices for things (SMS, hardware, etc.)

Making the most out of all that data: leveraging existing small and big data to improve health outcomes (9:00 AM - 10:15 AM)

Attendees: Swetha and Mary Jo (late)

Presenters: Asif (Living Goods), Gabriel Krieshok (Abt), Gina Assaf (SoukTel), Jacques de Vos (Mezzanine Vodafone), Vidya Mahadevan (BlueSquare)

Overview

The digital health ecosystem is increasingly interrelated. Countries are confronted with an abundance of data. Adding big data sources like population and geographic data creates an unprecedented amount of data available... but is it being used to it's potential? There is an opportunity to make smarter use of existing data to minimize data collection, gain a comprehensive, real-time understanding of what is happening now, and predict what to expect next. However, much of it remains untapped and unused. This panel will focus on innovative ways to analyze existing data from multiple sources to provide new insights and support decision-making, grounded in case studies and field experience. It will present practical tools and techniques to optimize how we use routine, program, and private sector data, as well as opportunities to use machine learning. It will also explore data governance challenges-including negotiating data sharing agreements and addressing data reliability.

Introduction

Innovations – are high in cap ex
Even in the US – unwanted hospitalizations – preventable
Diverse data being gathered and integrated = challenging because data is in silos
It’s about optimizing systems – to work together

Souktel

Creates end to end digital solutions for USAID, DFID, UN implementers

Cross platform services In frontier markets / crisis zones

Areas of intervention

beneficiary engagement (feedback loops)

data management (case management analytics),

monitoring, evaluation & learning

Core solutions: digital strategy advising – go to market outreach / content delivery / ME&L and Analytics – data collection and indicator tracking

User centered design approach

Identifying data user needs (what is possible vs what is easy to integrate)

Showing use cases and seeing users respond is helpful

Use cases

Rural clinic patient reporting platform Vanuatu MoH

Made data available to aid workers via SMS

Searchable web interface

M&E of Service of Aid Delivery

Providing service to the donor (client) and the implementer (the one with the reporting needs)

Medical Supply Chain management – IRC

Scannable QR code, to track inventory

Learnings

Start with the Use cases and users that need the data

Identify new channels to disseminate and use the existing data

Prioritize by level of effort to integrate data with impact

Plan for bureaucracy of access and effort in cleaning and integrating

Use of AI / Machine

Predicting malnutrition cases

Predicting famine related to food insecurity

Medical supply inventory management

Use of AI / Machine - Learnings

These are difficult to sell – smaller projects are user to sell

Large sample required, data quality and structure (integration, standardization) is important

Requires a change in staff skills

BlueSquare

Founded in 2012 / based in Belgium
Work with health data systems – MoH, WHO, NGOs, Pharma cos,
Approach
Pulls data from DHIS-2
Runs computational algorithms
Feeds data back into DHIS-2
Sits on top of work around DHIS-2 stupid proof prettier than DHIS-2
DHIS-2 to DHIS-2 for multiple DHIS-2 instances multi-system environment

Working with existing systems
Results based approach RBF main platform and work based on DHIS-2
DataViz

Data cleaning
Do we know where data comes from
Facility Level Data: Start there!
Look at the differences in lowest and highest unit of hierarchy
Build in translations (CS = Centro de Saude = Just health facility name)
That’s where everything data
How to consolidate multiple data sources
Hierarchy of errors

Siloed environments – data is not interoperable
Overlap between govt work / data collection
Use case: DRC → Fragmented data environment

Data Analysis Process
Clients = ministries
Pulling in QoC, Service delivery, Burden of disease, World Population
Take a dashboard data, combine with the above “distance to health centers for women of child-bearing age” --> more robust

Get data element, data element group, data systems
“All the best data available in one place”
HMIS data overlaid with benchmarking against other external data
Dashboards made publically available to make sure that partners to data

Abt

Overview

Global research and technical research firm private sector firm work in GH and PH

3 key concepts

Maslow’s hammer

Don’t be constrained by current tools/approaches/techniques. Ask “why” to understand your problem and then work inwards towards tools

If all you have is sharepoint / tableau you automatically think of those -gGet out of the thinking that we only have one tool
Think of the higher level problem
Data sources: documents + Websites + Social Media data stores + data analysis analytics / classification / modelling

Modularity

Think in terms of components of a system that can connect independently.
Modules can be replaced, upgraded, transformed, understood, and sustained by different people (stakeholders).
Systems components and how they may be separated and recombined – with the benefit of flexibility and variety in use
Helps to build sustaining systems and processes - be software agnostic

G.I. Joe Fallacy

Knowing doesn’t magically lead to change.

Data Dashboards aren’t a panacea. You have to think about the data *use*, your users, your problem

Mezzanine - Vodafone Mezzanine

Overview

Industry solution – for health care – South Africa based
Translate technology benefit into an economic benefit – data at the core of answering this question
Value of data is different, but you can get different data points

Process / continuum of data use
Data Driven Decision Making
High ROI: High value / high impact vs low complexity (logistic, technology, political) complexity is a weighted score
Use case:
E.g. Sensicardiac: stethoscope to look for congenital heart diseases lots of investments but only 8 / 1000 kids have this issue. This is not high ROI… clinical value, complexity was not considered
E.g. Use of mobile to increase stock availability mobile reported stock on hand on weekly basis by nurse triggers order from depot once a week do the stock, capture data (transferred, intake, on hand, issued, wastage) high value, low complexity

Monitoring
Control
Optimize
Automate decision-making

Think about: How can you take data point and support decision making across different contexts? E.g. Stock data:

MoH
Pharma companies
Logistics company,
Ministry of Finance / Treasury

When you take the same data and target multiple stakeholders, you will see ROI.
Data Use

We use about 5% of the data used for decision making. Make sure the data is used for decision making.

Questions
Gina: information architects think about how to present data in ways in which people can actually consume. There are keys to how health data is accessed because there could be a problem with that.
Below below is how teams are structured in private sector

Question: How much data can users actually consume?
Question: How should data teams be structured?

Data Scientist + Data Engineer
Domain Expert
Designer
Software Engineer

Tips:

Hire data engineer for cleaning / standardizing, so data scientist can analyze!
Ensure there are representatives of your customer on the team (so we know what question to ask)

Question: how do you decide about the return on investment?
Jaques: bring in the users and ask the right questions. Understand the levers between the different stakeholders. What is the objective and long-range view. How does this information really inform decision makers. What does not work… I want to sell a solution and than later on ask to sell the data later on it. Key point: pitch the model early on with the stakeholders.

Question: Mobile access brought up misaligned incentives. What do you think of the GG and aligning donors around systems outside the domain?

Asif: we do auditing for clean/quality data.
Gina: data review
Jaques: quality of data will improve as you move to using it for optimizing and decision making reasons.

Digital Solutions and Medical Vendors-Improving Health Outcomes through Market-based interventions

Attendees: Rebecca & Swetha

Presenters: Richard Wright (Unilever), Abi Gleek (Every1Mobile), Ting Shih (ClickMedix)

Key Takeaways:

What is Unilever’s interest in medical vendors? Funding from DFID to support social enterprises’ Unilever has shopkeepers and other ‘assets’ to support social businesses. They don’t do big health program interventions. This is NOT CSR. They help the businesses/grantees create proof of concept, and for Unilever its about growing their markets and getting people to use their products (more people washing hands=more clients)

Their goal is behavior change, to get people to use their product. This is very different from their usual business model.
Interacting directly with people to change behaviour is a big investment for a low margin product, and not necessarily a lot of return. Those you target might not even be able to afford soap.
Managing the relationship with the shopkeeper; eCommerce is the way forward.

Naijacare- enabling Medical vendors to strengthen delivery of primary healthcare services (PPMVs)

PPMVs are crucial for delivering healthcare services in Nigeria
HCD workshops in Lagos revealed that the vendors are not interested healthcare projects- they just want to grow their business. Iterate based on user needs. Need to leave lots of time for the design process--better for ‘fail small’ a few times and get the design right before scaling
Their interests are learning business skills, get bulk discounts, and get access to credit
eLearning on business, financial literacy, allowing them to connect with peers/business experts, allow them to apply for credit, and voucher system to help drive sales
Ex: training them to use RDTs for Malaria (helps them improve business and reputation, and improves health outcomes)
Developing a digital ordering system of over the counter medical products--linking them together in group buying scheme (this includes Unilever products)
The platform also plans to scale geographically beyond Lagos
Long term: engagement is first priority, because any possible long term revenue streams will depend on PPMVs using the system

ClickMedix- mobile platform to scale healthcare services

Started as a telemedicine platform
The telemedicine exposed caregivers to lots of new diseases and builds their capacity
Ex: cervical cancer: links the treatment, diagnosis, and followup over digital platform-sometimes
Partners include medical specialists (US-based); governments, training health workers (certification programs);
Private sector partners interested in growing their market-->better, cheaper, faster services

Partnerships with private sector: DFID & Unilever- Unilever structured the partnership to include others who can supplement expertise that they done have (Vodofone, Mastercard, etc.). Organizations can join at a project level or program level (for general interest).

Thinking about patients as consumers and people who are educated
Think about what people ‘want’ in order to drive the ‘needs’ (you need to wash your hands but entice people with what they WANT to do it)
Design with user needs, perspective and aspirations at the heard. You can’t know what that is unless you ask them. Building user empathy; what will be exciting/engaging for them?

Learning as we go: Adaptive Management for digital health

Attendee: Mary Jo

USAID Accelerate (Siobhan Green): attempting to integrate the USAID mission portfolio to be focused on behavior change across the silos. Key challenge is around collecting data to understand if the behavior changes are happening or not. Are the investments making the desired impact.

Was designed from the beginning to be innovative and perhaps will fail along the way.

Amy Green:

Had to understand that there is never a finished “bot”, it will always be a work in progress.
Each phase has an overarching research question and built it out in sprints.
Did not initially build in a way to say goodbye so from testing they quickly realized we had to build a way to say goodbye.
How did we gather data?

Inflow questions (worked)
Surveys (did not work)
Quizzes (worked)

Cannot ask data to extract information, users didn’t enjoy that.
Results: most girls self reported they didn’t have anyone else to talk to you.

Kelsey (JHU): mCare was a shift from directly controlling the delivery. mCare was using the public health delivery systems. mCARE-II (using OpenSRP) which is doing a lot of support around the life-cycle of care. Both supply and demand intervention.

Some key learnings

Challenging when you don’t “control” the human workforce
Philosophy of “supervision” should be aligned. In her example, the government viewed it differently than her institution. Folks were not empowered. There may not be replacement plan when government person is out.

Question: donor and MOH buyin?

Kelsey: WHO convened the approach of abandoning the pilots and work together to build a global good. MoUs signed with government but there were timelines on there so no issue with delaying.

Question on CLA (collaborate, learn, adapt) and Adaptive Management principles alignment.

Siobhan: they are aligned and are aligned into the process. Behavior change is based on triggers and understanding “what do we know”.

Merrick: Key thing is that there is intention behind the desire to be adaptive.

Siobhan: making adaptive concrete into making better outcomes.

Kelsey: flexibility are very important and will need to throw things out of the window.

Amy: focus and open-minded. Focus on what matters while not throwing things out the window.

Real time decision-making

Attendee: Swetha

Speakers: Jacqueline Edwards, Sherri Haas, Natalie Tibbels, Chancy Mauluka
Moderator: Emily Nicholson

Overview

Informed decision making starts with the availability of good quality, timely data. This session describes 3 initiatives that make data available in real time to inform decision making. In Malawi, UNICEF used ODK to implement a social accountability project to increase citizen voices for improved services in reproductive, maternal, neonatal, child, and adolescent health. In eSwatini, the HC4 project (implemented by JHUCCP) is using an online data dashboard of collected data to strengthen the capacity of local leaders to support community-led HIV prevention and response. And Medic Mobile has incorporated a family equity survey into their app to help influence CHW programmatic decisions and care coordination in the settings in which they work.

Medic Mobile

Medic Mobile – world class software for health workers
Software for community health - decision support and driving care
Equity Lens Pilot Analysis using the Equity Tool (Medic Mobile)
Goal:

Ensure that quality healthcare is reaching everyone
Measure equity - at the core of UHC
Relative wealth is important

EquityTool Details - basis for Smart Health App

EquityTool, supported by Metrics for Management (M4M) - a tested, community-validated way to identify which households fall into each wealth quintile (i.e., bottom 20%, next 20%, next 20% etc.), relative to the broader population of a particular country.
Free, low-tech, scientifically-validated tool designed to help managers get the real-time data on client relative wealth that they need to adjust and improve service delivery.

Smart Health App - How it works:

Partnered with Livng goods: Medic Mobile has partnered with Living Goods since 2014 to support their network of over 5,000 Community Health Workers (CHWs) across Kenya and Uganda with a custom, co-designed version of Medic Mobile’s open-source software, called the Smart Health app. Following discussions with Living Goods and the team at M4M, we decided to conduct a pilot analysis in three Living Goods branches in Kenya in 2017.
Embedded the EquityTool questions into a simple digital form that CHWs already used to survey families within the existing Smart Health application.
Easy integration: Able to obtain essential demographic information about the population served by 1,400 Living Goods-supported CHWs, including the family’s relative household wealth.
Use: By collecting this information alongside service delivery information and health data, goal was to support CHWs to improve and tailor care provided at the patient doorstep → strengthen workflow

Overview

Gain community view on health service provision

Use that “vox popli” to discuss with service providers

Also broadcast in radio

Problem

Malawi → the lack of accountability by duty bearers was identified as a constraint to health care.

Intervention

UNICEF launched a social accountability project in 2016 to increase citizen voice for improved services in reproductive, maternal, neonatal, child and adolescent health, using the Bwalo model.
Emphasizes citizen empowerment to demand quality services and to influence improvement in services and health standards,
Reach: 48% of the population in five districts.

Intervention components:
Poor referral systems – buy airtime, allocating fuel
Distance to health center – new clinics, improve infrastructure
Regulations → CHW stopped abusing health facilities

Information gathering: Through Bwalos, community representatives gather information to generate evidence on issues affecting demand and perceived quality of services.
Accountability: Information then presented to duty bearers at community, health centre and district levels where they demand actions and improvements in health services.
Escalation: Issues that cannot be addressed at the district level are referred to national platforms including the Parliament and the Ministry of Health.
Amplification: Escalation to national platforms happened through media that amplifies the community voices and through national forums organized by civil society representatives that utilize issues identified by the Bwalos as evidence for advocacy.
Example of issues:

Project challenge
ODK as a real-time data collection tool à helps in the continuum of data use

Not many phones available
ODK needs other packages e.g. SPSS

Precision Health and Service Delivery

Attendees: Vidya & Rebecca

Panelists: Matt Berg (moderator), Benjamin Winters, Kelsey (?), Guy Vernat

Key takeaways:

This session seemed to define (intentional or not) precision public health to be exacting use of GIS mapping for health service delivery

Examples included using GIS mapping and check-ins to establish a more accurate denominator (Akros and Jivita/Ona examples from Zambia and B’desh); leveraging ubiquitos biometric tools to create more transparency in service delivery, etc

Guy Vernat, Foundation Merieux USA

I-Lab: Connecting clinical Labs to Infectious Diseases Surveillance Systems in West Africa

Over a period of 12 months, I-LAb mapped all the lab sites in Senegal using WHO lab assessment tool, including location as well as devices available and types of diagnostics possible

-DHIS2 used to share reporting and analysis of data, but was not used for labs. I-Lab project implemented DHIS2 for electronic disease surveillance

-captured data re: lab locations, lab capabilities and created country’s first “Lab Book”

ACTION: any interest from AmosTaxi to connect?

Benjamin Winters, Akros

Zambia based company focused on developing surveillance processes and other health interventions and DHIS2 trainings

Worked closely on Macepa project with PATH and used “reactive case detection” with DHIS2 to map areas

Process management of campaigns (vaccinations, IRS, etc.). Looking for better M&E models, process improvement

Indoor Residual Spray- found more accurate way to measure/calculate coverage. ‘Denominator challenge’--Akros & Ona worked together to identify structures and guide field teams. mSpray takes maps, puts them into a device, and helps teams know where they need to go to spray and record data. Creates breadcrumb dataset. Using mSpray caused teams to find more structures and spray more. This instance was a success, but what we want is overall govt improved performance

-learned that the mSpray concept had applicability across many types of campaigns

Call to action: if we want our solutions to stick over time, more investment into the data culture that funds these innovations is needed

Kelsey (?) standing in for Alain Labrique

Geo-spacial Innovations to Improve Equity and Measure True Coverage

JaVitA Study Area- households are mapped, women have ID number.

Community maps are usually simple, hand-drawn, and have unclear boundaries.
Gaps and duplications in frontline worker coverage became apparent when geospatially mapped
How do you attribute different levels of effort across HCWs- so they can give credit when due, and evenly distribute work (see how far they walk, how fast, and how long they stay at each place)
Working to identify denominators more effectively (how many people or structures exist in a certain area?)

-Tech has advanced faster than government structures have to support them.*this issue was raised in earlier governance session today as well*

-How can we leverage the private sector?

-What will next couple of years bring?

Impact of Digital Health Interventions on Data Use and Health Outcomes - Immunization Focus

Attendee: Vidya

Session objectives

ToCs state that there is an impact of digital health interventions on data use and health -- save/improve quality of life, save time, save money BUT the actual evidence lacks the rigor/stds to actually state their usefulness as fact

Today's session presents some of that better research

Garrett Mehl - present 2 reviews

Have contacted him for the presentation deck of his two reviews because actual talk was super choppy, hard to follow and full of tech issues

Jessica Shearer -- Dir, Health Systems Analytics and Technical Lead on IDEA

IDEA report now available on www.findyourfinding.org

They conducted a realist review of what works to improve the use of routine data in immunization decision-making with evidence from peer and grey literature of work done in LMICs. (Realist reviews look at whether something worked, why they worked and dig into mechanisms/contexts where they worked in order to scale up) This analysis resulted in an evidence gap map -- mostly greys (grey lit) and few blues (peer reviewed). They categorized the evidence as "high certainty of evidence, some certainty of evidence, low certainty" but a major limitation is that they did not use the same set of outcome or impact measures for all the evidence they reviewed, instead using the publication's own interpretation of success in their classification.

Takeaways:

Interconnected strategies get better results -- interventions focused on skills and capabilities along with technology far more effective than tech interventions alone
Systemizing data use leads to long-term success -- get the data use piece down (how are they using it, when, what is the process it's going into, etc) along with the actual tech deployment
Difference between LMIS and HMIS usefulness --- strong evidence from review that LMIS systems have improved data availability and use and low certainty of evidence of HMIS systems doing the same, particularly on data use front
Assumption going in had been data quality will lead to better data use but the review found the opposite --- that data use leads to better data quality.
Across the breadth of studies reviewed, the HF was forgotten completely in the data use side -- only focus for HF was on improving data quality but not about establishing feedback loops or demonstrating value add of the data being collected. This is a missed opportunity they wanted to highlight.

Nargis Rahimi, Shifo Foundation

Conducted scoping review, did not put a time period barrier, looked at all peer-reviewed publications looking at what factors contribute to poor data versus good data, and facilitations of what interventions work to improve quality.

Identified 1000+ articles on the topic but only 25 made the cut!

Takeaways:

A big contributor to poor data quality is when a promising pilot ends up scaling too quickly or prematurely. Evaluate critically, remember that your pilot has to be sustained by existing health system structure -- but somehow every project they studied got this wrong.
--> vidya side note: I've been reflecting on this a lot given our experience with SELV -- going from 4 provinces to 11 in less than a year and with same # of staff to support…
Each implementer has a responsibility to better inform our own work and action based on what we know -- not the just implementation of a new project but the overall strategy, too. There doesn't seem to be that self-reflection in the reviews studied -- same implementing partners did X in one country, wrote up their learnings and 3 years later, did the same in Y country and nothing had changed in approach or strategy -- why?
Add an evaluation component from the get-go to your project -- ensure following research protocols and collect evidence accordingly -- that is how the poor evidence base in this area will be improved

Country Experience with Strategy and Governance (10:30 - 11:45am)

Attendee: Rebecca

Key takeaways

Malawi: there is fragmentation of mhealth interventions in countries. There is a need for streamlining interventions, and use applications that span across program areas. Standards are needed for hardware, software, hosting, training, etc.
National assessment-->draft of plan/roadmap-->governance strategy-->funding
Advice: strategies need to speak for health sector, not just the technology
Ethiopia wants to focus on using standards and interoperability; specific TWG for this
Kenya, TZ, and Ethiopia presented on their digital health plans/roadmaps and priorities
What is a strategy vs Policy? Policies don’t explain the ‘how or why’, it’s just a law. Strategies contain specific steps, which need to be budgeted for and measured (indicators). ‘Vision without resources is hallucination’’.

How do you develop policy to support a strategy? Countries need support on this. They request the global community to create a blueprint for this

When moving toward digitization, it is important to provide tools to improve culture of data use. If they aren’t used to using performance data to make decisions, they need to be taught to do so. Can’t assume digitization alone will solve problems. Data must be appreciated.
Interoperability: Kenya has enterprise architecture, provides interoperability. They have a mobile service centers for people who don’t have good mobile connectivity. Kenya has a data depository/dashboard (with support from WHO).
What are top 2 needs for capacity building in your govt?: Malawi- use of data at the facility level, TZ-health informatics techniques for graduates at various levels, they need people to train the service providers to generate and use information; infrastructure/ support for tele-medicine; Ethiopia: understanding of global standards/interoperability
Call to action: mHealth is here to stay, we need to move forward with the tide and improve our underlying systems to embrace digital health innovation