Contents

Global Digital Health Forum 2018

When: December 10 - 11th 2018

Where: Washington DC

Attendees: Rebecca Alban (Unlicensed), Vidya Sampath, Swetha Srinath (Unlicensed) and Mary Jo Kochendorfer (Deactivated)

The following is a joint trip report. Please note that the session notes are quite detailed and some are rough notes. Feel free to leave questions and comments for the authors.

Overview

Team spoke on two panels (Mary Jo and Vidya), presented two posters (Swetha and Rebecca), attended lots of sessions, and crammed in lots of meetings in two very full days. Some photos and table of Contents with LINKS below!

Rebecca presented a poster on OpenLMIS.

Agenda

Here

Follow up

Follow up for OpenLMIS Stewards

• BAO: Rebecca to send Trusted Partner information to Steffen and BAO systems. - DONE
• Zambia: Mary Jo to bring up SMSforLife to JSI team and understand what their thoughts are of the application and potential use of it or integration with it in Zambia.  Rebecca to follow up and see if the JSI team would be willing to share more details on the online training developed with the community. A potential addition to the OpenLMIS Implementer Guide.
• Bahmi: Follow up with Steven (swanyee@intellisoftkenya.com) about an opportunity with MedSource. - DONE
• Jembi: Mary Jo Kochendorfer (Deactivated) to follow up with Chris about ways we could potentially work together. Mary Jo met will him after our panel and will continue the conversation in 2019.
• Mezzanine: Mary Jo Kochendorfer (Deactivated) to send the Zambia RFI. - DONE
• Mezzanine: Mary Jo Kochendorfer (Deactivated) to ask for more information and the source code is on SMSForLife.
• TZ/Alpha: Mary Jo to follow up and obtain the diagram he created for the eHealth architecture for Tanzania. - DONE
• Miguel Sitjar (Palladium): Rebecca to send follow up email providing him with community resources and contact information for SolDevelo that he and his colleagues can use for potential implementation Guatemala- DONE
• Meaghan WHO: Rebecca to send the follow up email CCin Vidya and Mary Jo. We will connect about Digital Health Guidelines and  Ask for the contact information for the regional ICT point people Derrick, Mr. Ba, and Mark. - DONE
• I-TECH: Mary Jo Kochendorfer (Deactivated) to follow up on the Notice C efforts and connecting with the OpenELIS product. She will reach out to Jan Flowers, Casey and Joanna. The group plans to meet in the new year. https://proposals.digitalsquare.io/91
• Shifo Foundation: Mary Jo Kochendorfer (Deactivated) met Nargis Rahimi and will follow up to learn more about SmartPaper to see if it could be useful for the OpenLMIS implementations. Brandon has also been connected with Nargis. Plan is to schedule a demo in the new year. Potentially very exciting.
• Abi Gleek (Every1Mobile) Rebecca Alban (Unlicensed) to connect with Abi Gleek, who implements the  Naijacare app (see session notes below) to learn more about their digital ordering system and e-learning modules that they have built into Naijacare

Presentations

Here

Session Notes 

Facilitating Collaboration to Accelerate Scale and Improve Digital Health Global Goods

Attendee: Rebecca and Vidya

Presenter (panel): Mary Jo Kochendorfer (Deactivated), Matt Berg (Unlicensed), Amanda BenDor (Unlicensed), Chris Seebregts (Jembi), Steven Macharia (Bahmni) 

Description: Digital Square, an innovative co-investment global program led by PATH aims to serve as a convener in the digital health community, bringing together practitioners to share their work, lessons learned, and future needs. This helps organizations to work together to implement digital health tools that are adaptable to different countries and contexts, also known as Global Goods. To foster transparency, collaboration and synergy, Digital Square uses an open application platform including community feedback to align investments in digital health systems. The open application process provides a unique opportunity for practitioners to learn key details about, and comment on, each other’s tools and implementation plans. This session will describe the innovative open application process and feature global goods awarded through Digital Square including the OpenLMIS, Community of Practice, OpenCRVS, OpenSRP, and Bahmni.

Digital Square- lives at PATH but is a consortium of donors. They are a funding mechanism, and coordinate digital health investments

Open Proposal process- community feedback from peer review committee and governing board, comments, and create coalitions. Transparent process

Bahmni-open source Hospital Information System (HIS) and EMR; does not require custom software development; about 50 implementation

Jembi- OpenCRVS (civil registration and vital statistics); register children, deaths, big unmet need; works with FHIR 7, HopenHIE, and DHIS2. Supports OpenHIE architecture; can interoperate with low demand service like civil registry, with a low demand service like vaccination

OpenSRP- focus in care at facility and community level. Integrates with national health systems like DHIS2. Does Client ID, client management. OpenSRP community   Canopy reporting stack is result of DigitalSquare investment

?What are challenges with Digital Square open proposal process?

• Benefits- standards are valued, collaboration with other open source is encouraged. Peer review was helpful. Digital Square has been helpful for some in terms of opening up other funding too. Organizations appreciate the validation as a ‘global good’.
• Challenges- it is a new process; smaller communities might have trouble keeping up with the administrative lift. Sometimes the amount of money is now known, so that can cause confusion and is hard to plan for. Some organizations don’t necessarily want the proposal process to be so open (don’t want to share their idea).
• Can be a lot of work for not a lot of money
• Amanda’s role is to ensure process is clear, easy for participants. Have to divvy up the pot of funds to spread across multiple global goods
• 5 million dollars invested in 27 global goods initially, went up to 12 million
• Global health funding in general is not directed toward developing core technology. This funding model is better than giving directly to the implementers to decide what to do with it . Longer-term funding can be a gap
• How to harmonize what you are doing at the program level, and balance it with global goods- make it easier for people to build into the health eco system
• Fragmented donors and fragmented funding can make it hard to develop global goods, Digital Square and DIAL are easing that situation
• There is no business model to sustain things -like having a product manager. Its important to think about blocking off specific capacity
• Could think about building in some mechanisms for countries/implementations to pay a licensing fee of some kind. This could encourage sustainability to fund ongoing maintenance of the global good. Software is a ‘leaky boat’ that always needs care
• Suggested improvements for proposal process: support for intellectual property and general community building, also the other ‘soft’ thing around OpenSource. People tend to volunteer for actual coding, other things such as management, not so much
• Suggested investment in Dev ops-
• Connect-a-thon and other capacity building to teach about FHIR and standards

Impact of electronic systems on health service delivery in Zambia

Presenters: Wendy Bomett (JSI Zambia), Chris Opit (Unlicensed) (JSI Zambia)

Attendee: Mary Jo Kochendorfer

• Provided highlights of the implementation of eLMIS and Facility edition.
• Wendy went over the business processes.
• Receiving
• Inventory management
• Dispensing
• Chris will talk about the impact of an electronic systems. Emphasized the importance of having manual systems in place prior to rolling out the electronic system.
• Automated reports generated from electronic systems
• Reduction in workload at SDP and Central level (no specific figures showed)
• Improved efficiency and accountability (due to increased in reporting rates)
• Commodity availability - before the system they can see what facilities have overstocked items to “share” while awaiting their consignment
• Timeliness (78% fully automated 66% that are not fully automated)
• Workload decentralization has helped with data entry workload
• District facilities still have the most workload but central is down to zero.
• Improved Data use
• Tracked of number of sessions (not sure if they mean login sessions)
• Improved report timeliness by targeted training and championing. Having a system itself can only improve so much. Still needs training and human prioritization.
• Reduced wastage: eLMIS cuts expired drug inventory in hospitals.
• Uses the system each monday to check the stock status to know if they have overstocked items and can redistribute. Showed a graph of reduction in ARVs (from 1.5% - 0.2%)
• DEMO
  • Select the product source, program area, Dispatch Number (from the dispatch Note)
  • Select each product and batch information
  • Allow for entering in their own batch numbers
  • Then shows the stock control card which can be printed.
  • Select date, program and receiving node, person
  • Issue voucher is generated for printing
  • Select program area
  • Have to add each product one by one
  • Quantities and remarks.
  • Client ID, Gender, DOB
  • Seemed to only be for ARV clients
  • Goal is to dispense in under 1 minute
  • They enter the number of days needed for the medication. The system says how many bottles.
  • Subsequently, the daily registry is updated.
• Facility Edition - the web version was showcased.
• Received products
• Issue to the dispensary
• Adjustments
• Physical Count
• Dispense (doing a bulk upload from the EHR)
• Discussion:

Zambia is looking to automate down to the community level.

• What electronic systems do you use for supply chain management in your country?
• What are your pain points?
• Questions:
  • Started work on that. Collaboration is very difficult. Interface is working in two - three more facilities. Now are testing the web-enabled version of facility edition.
  • Chris: yes… but didn’t clarify and mentioned paying attention to security
  • Wendy: yes. Needs to rolled out by the government. No report no product principle helps move to systems.
  • Data quality and completeness. There are data validations built into the system so it is harder to “quickly submit” anything. You have to true up things.
  • Chris: big issue. When we developed the training program, we looked at sustainability. How to continue without us re-training? We train people on the job (hands-on training). If you go to the facility, you can train everyone in the facility. Facility then takes over in training interns and new people.  District health information officers will also train facilities.
  • Wendy: there is an e-learning module for them to print out a certificate. Annual meetings to discuss the data and will hold a training on the 3rd day or something.
  • Wendy: we have a report who can look at the adjustments which HFs are making. Particularly with facilities that are automated the entire way.
  • LAN FE system
  • They wouldn’t abandon because built on manual processes
  • Sustainability plan for the ministry to own.
  • Everyone wants something to track transactions. Phones won’t work because everyone has at least 100 products.
• Have you automated pharmacy dispensation?
• Are you interoperable with SmartCare? (from CDC - Dan Rossen)
• Are you following OpenHIE principles?
• Did we face transparency challenges?
• Challenges with uptake of the system at the facility level?
• Sometimes HF report on time but then they have to go back and resubmit?
• Who owns the data?
• MSH Question: you many facilities … how do you manage staff turnover/attrition and training?
• WHO Humphrey Question: Issue with leakages (paper and what is on the ground discrepancies). Are there issues with redistribution (which allows for more opportunities for leakages).
• WHO Question: What about internet access?
• If JSI was to walk out today, would the system still be used?
• Question: For low-throughput clinics, have you thought about using mobile?
• A system should be there to help a ministry not lose money

Opening Plenary- How funders are operationalizing digital health principles

Attendees: Rebecca, Swetha, Mary Jo

• Emphasis from CDC and USAID on the importance of them shifting from supporting bespoke tools to self-reliant tools and Global Goods
• GIZ mentioned Digital Square and DIAL as platforms that they would be interested in leveraging
• In order for donors to follow these principles, countries actually need to have a digital health strategy. So there is a call to countries to create these plans in order to get funding in this area

Strengthening primary health care through digital tools

Attendee: Swetha Srinath (Unlicensed)

Speakers: Maryanne Mureithi; Jamil Zaman (Bangladesh), Ephrem Lemango

Moderator: Uju Aderemi

1. Medic Mobile [Ethiopia]

• Focus of intervention:
• Care coordination / referral network
• Overview:
• Community health innovation network – partnership with living goods
• Integrate community activities – health facility care
• Increasing reach through technology
• Why Referral
• High priority cases
• Time-sensitive
• Closing the loop
• Referral pain points
• Lack of referral tools / loss of tools  [paper systems – supply chain]
• Accuracy of reporting
• Px / Patient experience [long lines]
• MVP Design
• Digitized referral systems
• Referrals trigger tasks
• Referral targets
• Facilitating communication
• Challenges
• Internet communication issue
• Performance of app
• Keeping the workers engaged
• Involved them in design process

1. Mobile Job Aid for H.E.Ws [Ethiopia]

• Focus of ppt:
• RMNCH
• Overview
• Health Extension Program
• Intro: HEP is one of the strategies adopted by the government of Ethiopia to achieve universal coverage of primary health care among rural population
• Goal: to create a healthy society and to reduce maternal and child morbidity and mortality rates.
• Overview: Program that is deeply rooted in communities, providing primary level preventive activities to household members. In addition to community activities, HEP also provides health post–based basic services, including preventive health services such as immunizations and injectable contraceptives, and limited basic curative services such as first aid and treatment of malaria, intestinal parasites, and other ailments. Case referral to health centers (HCs) is also provided when more complicated care is needed.
• Why Mobile?
• Coverage / equitable coverage
• Timeliness of visits / care
• Components of tool
• Automated  reminders and follow up
• Mobile job aids
• Leading to improved referral work flow
  
  
• App points of intervention
• HEW interaction with client: ANC 1 – ANC 4
• ANC 1 – sent to health center for first one, then sent back to community
• 2nd, 3rd, 4th sent to community / HEW
• After delivery, HEW notified to provide post-natal care

• HEW Application
• Provides info on location where visit should take place
• Provides timeline for timeliness of visit
• Provides referral management etc.
  
  
• Status / Progress
• Issues: Capture at health facility level when delivery happens – needs to improve
• Political commitment: Needs to start from minister level
• Risk: Intermittent inaccessibility / increasing technical complexity / changing and expanding vision and scope
  
  

1. eMIS (Save the Children Bangladesh)

• Focus of intervention:
• eMIS
• Partnership:
• MOHFW
• Problem
  • Included supervision, though cumbersome: Paper-based system included a supervision process to ensure that FWAs and HAs were visiting the households and correctly completing the forms, but it was cumbersome to
    Supervisor role: Supervisors required to visit the households or satellite clinic that the HAs and FWAs supported to verify the data collected and to review the registers and reports.
  • Lack of supervisor feedback: Under the paper system, FWAs and HAs did not receive direct feedback on their performance in completing the required forms
  • Data use not regular: Under the paper-based system, FWAs, HAs, and supervisors did not regularly use the data they collected to improve quality and coverage of services. The information collected was primarily reported to the higher level,
  • Only option for referral: The only way to know whether a client completed a referral was through self-reporting during follow-up visits, which could be occur between two and four months after the initial visit. Facilities to which the clients were referred did not necessarily keep records of referrals, and the reporting mechanisms for referrals were weak.
• Inability to track patient along continuum of care
• Reporting delay / slow and error prone
• Lack of sync
• Paper-based system
• Digital Application – eMIS
• Registers needed to be unified / simplified (across continuum of care)
• Indicators (for appropriate stage) included
• Digitization process
  • Client unique identity – pregnancy registration
  • Provides prompts and alerts based on protocol
  • Protocol embedded into the tool
  • Longitudinal health records
  • Stratify clients by risk type
  • Schedule work plan for providers
  • Monitor performance of healthcare provider  
  • Assess health facilities
  • Manage health commodity inventory
  • Data Services  
  • Non routine data collection
  • Automated analysis to generate new information
  • Map location of facilities
  • Reduced workload of HAs and FWAs: eMIS has significantly reduced the workload of HAs and FWAs and errors that were related to transcribing the data to different paper forms.
  • Time spent compiling info: When using the paper-based system, FWAs and HAs would spend one to two days each month compiling the monthly HIS report. With the eMIS, they can press a button and the report is generated instantly.
  • Better tracking of patients: Unique health identification system (master client index) established in the eMIS through a population registration system has improved quality of care by allowing health workers to track individuals over time and ensure the continuum of care  At the facility level, service providers can pull up an individual’s information by using the unique health ID and retrieve all stored data from the population and service databases. FWAs and HAs can also retrieve services provided to their clients as well as check on referrals using their tablets with the unique health ID.
  • Service mapping: Dashboards generated from the data collected on the tablets have helped district and national-level managers plan services to ensure the continuum of care. A manager can pull up services by a particular provider to view data by date. This reduces the need for supervisors to travel to the field to check all aspects of quality of care to determine where improvements are needed.
• 1. Provider System
• 2. Health System Managers
• Results with eMIS

1. Questions

• Was paper replaced in Bangladesh?
• Double reporting – in initial stages; when reliance on digital increased, paper was replaced in Bangladesh
• 2-3 months lead time: for HCPs to get used to smart phones
• Are these tablets in Bangladesh / Ethiopia – real time offline?
• Ethiopia: moving towards tablets, which should work offline
• Bangladesh: All data is continuously synching – when internet is available, synch happens in background
• OpenSRP for case management – how to make sure you refer without duplication
• CHWs refer patients within app  when activities initiated, they have to go through app to trigger referrals  all activities are coordinated using the platform, and CHW has to be registered
• Data Privacy?
• If we want to check if follow-up is done, we only display CHW performance, as opposed to patient status / data
• Bangladesh: govt doesn’t have legal framework RE: data privacy – working with govt based on paper based privacy policy – a) Also patient identity based on number, not patient name pre-referral and b) data only becomes available to next level provider IF patient is referred.
• Linkage to EMR
• CHWs refer patients within app  when activities initiated, they have to go through app to trigger referrals  all activities are coordinated using the platform, and CHW has to be registered
• CHW Registering – at health facility + community?  
• CHWs – “hashing”? non deterministic
• eReferral – patient level challenges – any cases of where patients referred but they didn’t show up?
• Digital solutions are not the only solutions
• This platform encourages continued community worker – persistence – ensures patients get care

WHO's 1st ever Guidelines for Digital Health Interventions for HSS

Attendee: Vidya

Panelists: Garret Mehl, Tigest Tamrat, and Smisha Agarwal

Notes:

PATH, as the support org to WHO, convened a session with the WHO team that has put together the very first WHO Guidelines on Digital Health Interventions for HSS.

Team consists of Garret Mehl, Tigest Tamrat, and Maeghan Orton (latter two involved with early CCPF work)

The guidelines for formally approved by WHO board this week and will be made publically available early next year. PATH is helping put together an interactive website to accompany the print publication. The guidelines will also have helpful language/visuals to show how these interventions map to Universal Health Coverage (UHC) goals.

This work is part of the larger Digital Health Atlas compendium that this team is also leading (a WHO global technology registry platform).

The team first developed "Classification of  Digital Health Interventions" and the Guidelines map to this classification.

The third deliverable is a "Planning and Costing Guide for Digital Health Interventions" which is set for release next Spring.

Monitoring and Evaluation of existing digital health interventions:

The team highlighted the usefulness of the mERA checklist which has been around for a couple of years now. For impact evaluations, they recommend the following study designs:

Controlled before and after;

Stepped wedge RCT;

Interrupted time series studies

More information on mERA here:

https://www.bmj.com/content/352/bmj.i1174 (paper summary)

https://www.researchgate.net/profile/Amnesty_Lefevre/publication/311738723_Monitoring_and_evaluating_digital_health_interventions_a_practical_guide_to_conducting_research_and_assessment/links/5858cdf908ae64cb3d47f7d3/Monitoring-and-evaluating-digital-health-interventions-a-practical-guide-to-conducting-research-and-assessment.pdf (detailed assessment guide including the checklist)

Takeaways:

• OpenLMIS should start folding in language about both principles of digital development as well as the Digital Health Interventions classification and guidelines in our future proposals and when talking about our current work.
• Tie efforts to UHC when possible
• OpenLMIS team (see follow-up/action section) is in touch with the WHO team about how to include the Implementation Toolkit in the interactive website that PATH is working on.
• Reference the mERA assessment guide for future digital health work

Putting the Patient in Charge- new tools to help clients manage their own health data

Attendee: Rebecca

Online self-administered screening tool for improving TB detection among students in Ethiopia (MSH)

• In Ethiopia, they have case-finding issues. ⅓ cases are undiagnosed. Targeting universities. Students were active partners in the initiative , made suggestions on how to implement, draft the self-screening checklist, and how do outreach, etc.

Online screening to be the first layer of assessment (rather than needing a HCW to examine each individual)

• The tool linked with CommCare for those whose screening showed suspected TB
• Targeted students on a free wifi system that they were already using (good use of resources); did targeted outreach in free wifi hotspots

Maternal Health in Liberia(D-Tree)

• Want flexibility to attend different health facilities throughout pregnancy/delivery
• About 80% of facilities in Monrovia are private. How do you adopt across a lot of actors; public and private. This can act as a central repository of data that bridges the public/private gap

Tuberculosis in Thailand (D-Tree)

• Migrant populations tend to lose touch with health system and default on

-->NFC token stores the client’s information. It can be a card, sticker, bracelet, etc. Client can keep that card and bring it with them. Its low cost (10x less expensive than biometrics).

• If the card is lost, the data is encrypted and card can be replaced. It is language agnostic
• Data is kspe on the phone and when there is connectivity it is synced with a dashboard

DOT Fertility App- to help manage fertility (Cycle Technologies)

70% of the world will own a cell phone by 2020

DOT fertility app uses user info and machine learning to track her cycles; help prevent or plan pregnancy

Ways to think about our own technology (OpenLMIS)

• Context- what problem you want to address. What is currently being used
• What are the reasons for non-use? (cost? access?)
• Usability- does it ‘delight’ the user?
• Evidence- what impact does research show? What usability research to you have to show users understand and like it?
• Scale up/Sustainability- what marketing is needed? What is your audience willing to pay? Are there business models (upgrade models, subscriptions, etc.) to consider?

Challenges of Innovation at Scale

Presenters: Wayan Vota (Digital Health Director, IntraHealth International) Jonathan Jackson (Dimagi), Dr. Iniobong Ekong, Clayton (Dimagi), Carmen Sant

Attendee: Mary Jo

Dimagi, Jonathan Jackson - the Digital health principles work for scoping (not at scale). Scope and scale are very different. Scope/pilot is what you get done on a certain budget for a certain set of users. Scale is really hard and procurement, training, data use, etc.

We didn’t have the initial challenging conversations at the beginning. We don’t have good frame-of-reference for those conversations.

For example, 130,000 scale was slowed due to procurement of hardware. Gave time for the conversation to happen because scope was deployed for 18 months before scaling.

Dr Iniobong Ekong - Nigeria (FCT eHealth Project).

ICT + Health systems = universal access.

Bought notebooks for doctors to support doctors in servicing rural. Access increased by 247%. All things going great.

Big challenge, had to still complete the manual forms and electronic systems in parallel.

Need to make sure the ART reporting is integrated into the EMR. Clear requirements and standards need to be in place. Now now new EMR can be rolled out without a clear ART module meeting the standards.

Top down doesn’t work. Standards are needed. Continuous evaluation of systems.

Clayton CTO - Talked about the Bihar scale up

130,000 CHW using CommCare.

5 years of working on the app before we scale.

Goal was to have non technical teams design and build the app.

242 servers to host the system right now.

What they had to focus on from 2017 was about tools for deployment. Not the actually application.

Carmen Sant (Research Assistant)

Health systems perspective. SMS for Life. 2009 over 5000 health facilities in TZ. Monitoring the stock outs. Ran for 4 years collecting the weekly data and then it was cancelled.

We thought it was really good from a technical perspective. Focusing on anti-malaria.

Surveillance without a response will not bring you to where you want to go. There wasn’t a plan for how to respond to the ‘stock outs’.  Who was responsible for the stock outs?

Perhaps the program didn’t involve all the stakeholders who needed to be there to realize this gap.

Pilot tech was inexpensive.

Costs at scale supposed a large burden for the government was quite large.

She overlayed the timeline from both the SMS for life and the government efforts.  There were parallel efforts and the SMS for life app didn’t change or wasn’t considered.

Focus of visibility, wasn’t enough. What about the action? A costing exercise would have been useful. Agile technology, program design, and project management would have been useful to allow for pivots.

Key Takeaways and Discussion

• In a pilot a system may “not be in scope”, but once you scale it will come into play. You don’t want to fill out two systems.
• How do you think the donor pression on national e-health architecture will play into things?
• Jonathan - is excited about the focus but it will be very challenging for companies reliant on project based funding. We will need to change how outcomes work since folks will not be excited to work on things which are not in your best interest.
• Dr Iniobong Ekong - there was a uforia around technology so folks were resistant of policy and regulations since it could slow the excitement of rolling out technology. Need to make policy useful and helpful.  Making it easier to roll out things with clear standards.
• CRS Marcus: How do we change the culture and begin to learn from our mistakes? We are worried that if we are not successful we may not get funding again.
• Clayton: There is still a root to failure. We sometimes continue to not allow things really fail fully.
• Jonathon: We gave ourselves a break for structural failure. We aren’t good at de-coupling failure. If no one is a owner to make decision, then you fail.  Governance issues will force failure. When we take a project with no funding after six months. We know it will fail.
• Waylon: Brought twitter to Cuba. Only had a specific amount of money for the cost. Pulled it. Thoughts on how to do that better. Is it a set up failure or what?
• Carmen: Both paying people to use the SMS in the initial roll out. The government wasn’t willing to pick it up. Also, the team wasn’t really learning as they go. RDT were added but no one new that they could use it now. Need constant training.  Why are we using “systems thinking”? Perhaps it is complex and we aren’t open to that.
• Question: Scope versus Scale. Sales mode you hide the complexity and then Scope phase you highlight the complexity. What is the honest broker?
• Jonathon: within scale (there is a huge amount scope in scale - like data security). That has to be driven by honest brokers around security Tech companies cannot ask for that given that they will be view. Let’s name bias. We are biased, doesn’t that mean your point isn’t valid. Can you create the right structures to answer the questions? Consultants should be determining if they should still be useful. Technology firms should choose the technology.
• Dr Iniobong Ekong - allow for learning.
• Clayton - you should account for market prices for things (SMS, hardware, etc.)

Making the most out of all that data: leveraging existing small and big data to improve health outcomes (9:00 AM - 10:15 AM)

Attendees: Swetha and Mary Jo (late)

Presenters: Asif (Living Goods), Gabriel Krieshok (Abt), Gina Assaf (SoukTel), Jacques de Vos (Mezzanine Vodafone), Vidya Mahadevan (BlueSquare)  

• Overview

• The digital health ecosystem is increasingly interrelated. Countries are confronted with an abundance of data. Adding big data sources like population and geographic data creates an unprecedented amount of data available... but is it being used to it's potential? There is an opportunity to make smarter use of existing data to minimize data collection, gain a comprehensive, real-time understanding of what is happening now, and predict what to expect next. However, much of it remains untapped and unused. This panel will focus on innovative ways to analyze existing data from multiple sources to provide new insights and support decision-making, grounded in case studies and field experience. It will present practical tools and techniques to optimize how we use routine, program, and private sector data, as well as opportunities to use machine learning. It will also explore data governance challenges-including negotiating data sharing agreements and addressing data reliability.

• Introduction

• Innovations – are high in cap ex
• Even in the US – unwanted hospitalizations – preventable
• Diverse data being gathered and integrated = challenging because data is in silos
• It’s about optimizing systems – to work together
  
  

• Souktel

• Creates end to end digital solutions for USAID, DFID, UN implementers

• Cross platform services In frontier markets / crisis zones

• Areas of intervention

• beneficiary engagement (feedback loops)

• data management (case management analytics),

• monitoring, evaluation & learning

• Core solutions: digital strategy advising – go to market outreach / content delivery / ME&L and Analytics – data collection and indicator tracking

• User centered design approach

• Identifying data user needs (what is possible vs what is easy to integrate)

• Showing use cases and seeing users respond is helpful

• Use cases

• Rural clinic patient reporting platform Vanuatu MoH

• Made data available to aid workers via SMS

• Searchable web interface

• M&E of Service of Aid Delivery

• Providing service to the donor (client) and the implementer (the one with the reporting needs)

• Medical Supply Chain management – IRC

• Scannable QR code, to track inventory

• Learnings

• Start with the Use cases and users that need the data

• Identify new channels to disseminate and use the existing data

• Prioritize by level of effort to integrate data with impact

• Plan for bureaucracy of access and effort in cleaning and integrating

• Use of AI / Machine

• Predicting malnutrition cases

• Predicting famine related to food insecurity

• Medical supply inventory management
  
  

• Use of AI / Machine - Learnings

• These are difficult to sell – smaller projects are user to sell

• Large sample required, data quality and structure (integration, standardization) is important

• Requires a change in staff skills

• BlueSquare  

• Founded in 2012 / based in Belgium
• Work with health data systems – MoH, WHO, NGOs, Pharma cos,
• Approach
  • Pulls data from DHIS-2
  • Runs computational algorithms
  • Feeds data back into DHIS-2
  • Sits on top of work around DHIS-2  stupid proof prettier than DHIS-2
  • DHIS-2 to DHIS-2  for multiple DHIS-2 instances  multi-system environment
• Working with existing systems
• Results based approach  RBF main platform and work based on DHIS-2
• DataViz
• Data cleaning
  • Do we know where data comes from
  • Facility Level Data: Start there!
    • Look at the differences in lowest and highest unit of hierarchy
    • Build in translations (CS = Centro de Saude = Just health facility name)
  • That’s where everything data
  • How to consolidate multiple data sources
  • Hierarchy of errors
• Siloed environments – data is not interoperable
• Overlap between govt work / data collection
• Use case: DRC → Fragmented data environment
• Data Analysis Process
  • Clients = ministries
  • Pulling in QoC, Service delivery, Burden of disease, World Population
  • Take a dashboard data, combine with the above  “distance to health centers for women of child-bearing age” -->   more robust
• Get data element, data element group, data systems
• “All the best data available in one place”
• HMIS data overlaid with benchmarking against other external data  
• Dashboards made publically available  to make sure that partners to data

• Abt

• Overview
• Global research and technical research firm  private sector firm work in GH and PH
• 3 key concepts
• Maslow’s hammer

• Don’t be constrained by current tools/approaches/techniques. Ask “why” to understand your problem and then work inwards towards tools

• If all you have is sharepoint / tableau  you automatically think of those -gGet out of the thinking that we only have one tool
• Think of the higher level problem
• Data sources: documents + Websites + Social Media  data stores + data analysis analytics / classification / modelling
• Modularity
• Think in terms of components of a system that can connect independently.
• Modules can be replaced, upgraded, transformed, understood, and sustained by different people (stakeholders).
• Systems components and how they may be separated and recombined – with the benefit of flexibility and variety in use
• Helps to build sustaining systems and processes - be software agnostic
• G.I. Joe Fallacy

• Knowing doesn’t magically lead to change.

• Data Dashboards aren’t a panacea. You have to think about the data *use*, your users, your problem

• Mezzanine - Vodafone Mezzanine

• Overview
• Industry solution – for health care – South Africa based
• Translate technology benefit into an economic benefit – data at the core of answering this question
• Value of data is different, but you can get different data points
• Process / continuum of data use
  • Data Driven Decision Making
  • High ROI: High value / high impact vs low complexity (logistic, technology, political)  complexity is a weighted score
  • Use case:
  • E.g. Sensicardiac: stethoscope to look for congenital heart diseases  lots of investments but only 8 / 1000 kids have this issue. This is not high ROI…  clinical value, complexity was not considered
  • E.g. Use of mobile to increase stock availability  mobile reported stock on hand on weekly basis by nurse  triggers order from depot once a week do the stock, capture data (transferred, intake, on hand, issued, wastage)  high value, low complexity
• Monitoring
• Control
• Optimize
• Automate decision-making
• Think about: How can you take data point and support decision making across different contexts? E.g. Stock data:
• MoH
• Pharma companies
• Logistics company,
• Ministry of Finance / Treasury
• When you take the same data and target multiple stakeholders, you will see ROI.
• Data Use
• We use about 5% of the data used for decision making. Make sure the data is used for decision making.

• Questions
  • Gina: information architects think about how to present data in ways in which people can actually consume. There are keys to how health data is accessed because there could be a problem with that.
  • Below below is how teams are structured in private sector
• Question: How much data can users actually consume?
• Question: How should data teams be structured?

• Data Scientist + Data Engineer  
• Domain Expert
• Designer
• Software Engineer
• Tips:
• Hire data engineer for cleaning / standardizing, so data scientist can analyze!
• Ensure there are representatives of your customer on the team (so we know what question to ask)
• Question: how do you decide about the return on investment?
  Jaques: bring in the users and ask the right questions. Understand the levers between the different stakeholders. What is the objective and long-range view. How does this information really inform decision makers.  What does not work… I want to sell a solution and than later on ask to sell the data later on it. Key point: pitch the model early on with the stakeholders.

• Question: Mobile access brought up misaligned incentives. What do you think of the GG and aligning donors around systems outside the domain?
• Asif: we do auditing for clean/quality data.
• Gina: data review
• Jaques: quality of data will improve as you move to using it for optimizing and decision making reasons.

Digital Solutions and Medical Vendors-Improving Health Outcomes through Market-based interventions

Attendees: Rebecca & Swetha

Presenters: Richard Wright (Unilever), Abi Gleek (Every1Mobile), Ting Shih (ClickMedix)

Key Takeaways:

What is Unilever’s interest in medical vendors? Funding from DFID to support social enterprises’ Unilever has shopkeepers and other ‘assets’ to support social businesses. They don’t do big health program interventions. This is NOT CSR. They help the businesses/grantees create proof of concept, and for Unilever its about growing their markets and getting people to use their products (more people washing hands=more clients)

• Their goal is behavior change, to get people to use their product. This is very different from their usual business model.
• Interacting directly with people to change behaviour is a big investment for a low margin product, and not necessarily a lot of return. Those you target might not even be able to afford soap.
• Managing the relationship with the shopkeeper; eCommerce is the way forward.

Naijacare- enabling Medical vendors to strengthen delivery of primary healthcare services (PPMVs)

• PPMVs are crucial for delivering healthcare services in Nigeria
• HCD workshops in Lagos revealed that the vendors are not interested healthcare projects- they just want to grow their business. Iterate based on user needs. Need to leave lots of time for the design process--better for ‘fail small’ a few times and get the design right before scaling
• Their interests are learning business skills, get bulk discounts, and get access to credit
• eLearning on business, financial literacy, allowing them to connect with peers/business experts, allow them to apply for credit, and voucher system to help drive sales
• Ex: training them to use RDTs for Malaria (helps them improve business and reputation, and improves health outcomes)
• Developing a digital ordering system of over the counter medical products--linking them together in group buying scheme (this includes Unilever products)
• The platform also plans to scale geographically beyond Lagos
• Long term: engagement is first priority, because any possible long term revenue streams will depend on PPMVs using the system

ClickMedix- mobile platform to scale healthcare services

• Started as a telemedicine platform
• The telemedicine exposed caregivers to lots of new diseases and builds their capacity
• Ex: cervical cancer: links the treatment, diagnosis, and followup over digital platform-sometimes
• Partners include medical specialists (US-based); governments, training health workers (certification programs);
• Private sector partners interested in growing their market-->better, cheaper, faster services

Partnerships with private sector: DFID & Unilever- Unilever structured the partnership to include others who can supplement expertise that they done have (Vodofone, Mastercard, etc.). Organizations can join at a project level or program level (for general interest).

• Thinking about patients as consumers and people who are educated
• Think about what people ‘want’ in order to drive the ‘needs’ (you need to wash your hands but entice people with what they WANT to do it)
• Design with user needs, perspective and aspirations at the heard. You can’t know what that is unless you ask them. Building user empathy; what will be exciting/engaging for them?

Learning as we go: Adaptive Management for digital health

Attendee: Mary Jo

USAID Accelerate (Siobhan Green): attempting to integrate the USAID mission portfolio to be focused on behavior change across the silos. Key challenge is around collecting data to understand if the behavior changes are happening or not. Are the investments making the desired impact.

• Was designed from the beginning to be innovative and perhaps will fail along the way.

Amy Green:

• Had to understand that there is never a finished “bot”, it will always be a work in progress.
• Each phase has an overarching research question and built it out in sprints.
• Did not initially build in a way to say goodbye so from testing they quickly realized we had to build a way to say goodbye.
• How did we gather data?
• Inflow questions (worked)
• Surveys (did not work)
• Quizzes (worked)
• Cannot ask data to extract information, users didn’t enjoy that.
• Results: most girls self reported they didn’t have anyone else to talk to you.

Kelsey (JHU): mCare was a shift from directly controlling the delivery. mCare was using the public health delivery systems. mCARE-II (using OpenSRP) which is doing a lot of support around the life-cycle of care. Both supply and demand intervention.

Some key learnings

• Challenging when you don’t “control” the human workforce
• Philosophy of “supervision” should be aligned. In her example, the government viewed it differently than her institution. Folks were not empowered. There may not be replacement plan when government person is out.

Question: donor and MOH buyin?

• Kelsey: WHO convened the approach of abandoning the pilots and work together to build a global good.  MoUs signed with government but there were timelines on there so no issue with delaying.

Question on CLA (collaborate, learn, adapt) and Adaptive Management principles alignment.

• Siobhan: they are aligned and are aligned into the process. Behavior change is based on triggers and understanding “what do we know”.  

Merrick: Key thing is that there is intention behind the desire to be adaptive.

Siobhan: making adaptive concrete into making better outcomes.

Kelsey: flexibility are very important and will need to throw things out of the window.

Amy: focus and open-minded. Focus on what matters while not throwing things out the window.

Real time decision-making

Attendee: Swetha

Speakers: Jacqueline Edwards, Sherri Haas, Natalie Tibbels, Chancy Mauluka
Moderator: Emily Nicholson

• Overview
• Informed decision making starts with the availability of good quality, timely data. This session describes 3 initiatives that make data available in real time to inform decision making. In Malawi, UNICEF used ODK to implement a social accountability project to increase citizen voices for improved services in reproductive, maternal, neonatal, child, and adolescent health. In eSwatini, the HC4 project (implemented by JHUCCP) is using an online data dashboard of collected data to strengthen the capacity of local leaders to support community-led HIV prevention and response. And Medic Mobile has incorporated a family equity survey into their app to help influence CHW programmatic decisions and care coordination in the settings in which they work.

• Medic Mobile

• Medic Mobile – world class software for health workers
• Software for community health - decision support and driving care
• Equity Lens Pilot Analysis using the Equity Tool (Medic Mobile)
• Goal:
• Ensure that quality healthcare is reaching everyone
• Measure equity - at the core of UHC
• Relative wealth is important
• EquityTool Details - basis for Smart Health App
• EquityTool, supported by Metrics for Management (M4M) - a tested, community-validated way to identify which households fall into each wealth quintile (i.e., bottom 20%, next 20%, next 20% etc.), relative to the broader population of a particular country.
• Free, low-tech, scientifically-validated tool designed to help managers get the real-time data on client relative wealth that they need to adjust and improve service delivery.
• Smart Health App - How it works:
• Partnered with Livng goods: Medic Mobile has partnered with Living Goods since 2014 to support their network of over 5,000 Community Health Workers (CHWs) across Kenya and Uganda with a custom, co-designed version of Medic Mobile’s open-source software, called the Smart Health app. Following discussions with Living Goods and the team at M4M, we decided to conduct a pilot analysis in three Living Goods branches in Kenya in 2017.
• Embedded the EquityTool questions into a simple digital form that CHWs already used to survey families within the existing Smart Health application.
• Easy integration: Able to obtain essential demographic information about the population served by 1,400 Living Goods-supported CHWs, including the family’s relative household wealth.
• Use: By collecting this information alongside service delivery information and health data, goal was to support CHWs to improve and tailor care provided at the patient doorstep → strengthen workflow

Overview

• Gain community view on health service provision

• Use that “vox popli” to discuss with service providers

• Also broadcast in radio

• Problem
• Malawi → the lack of accountability by duty bearers was identified as a constraint to health care.
• Intervention
• UNICEF launched a social accountability project in 2016 to increase citizen voice for improved services in reproductive, maternal, neonatal, child and adolescent health, using the Bwalo model.
• Emphasizes citizen empowerment to demand quality services and to influence improvement in services and health standards,
• Reach: 48% of the population in five districts.
• Intervention components:
  • Poor referral systems – buy airtime, allocating fuel
  • Distance to health center – new clinics, improve infrastructure
  • Regulations → CHW stopped abusing health facilities
• Information gathering: Through Bwalos, community representatives gather information to generate evidence on issues affecting demand and perceived quality of services.
• Accountability: Information then presented to duty bearers at community, health centre and district levels where they demand actions and improvements in health services.
• Escalation: Issues that cannot be addressed at the district level are referred to national platforms including the Parliament and the Ministry of Health.
• Amplification:  Escalation to national platforms happened through media that amplifies the community voices and through national forums organized by civil society representatives that utilize issues identified by the Bwalos as evidence for advocacy.
• Example of issues:
• Project challenge
  • ODK as a real-time data collection tool à helps in the continuum of data use
• Not many phones available
• ODK needs other packages e.g. SPSS

Precision Health and Service Delivery

Attendees: Vidya & Rebecca

Panelists: Matt Berg (moderator), Benjamin Winters, Kelsey (?), Guy Vernat

Key takeaways:

This session seemed to define (intentional or not) precision public health to be exacting use of GIS mapping for health service delivery

Examples included using GIS mapping and check-ins to establish a more accurate denominator (Akros and Jivita/Ona examples from Zambia and B’desh); leveraging ubiquitos biometric tools to create more transparency in service delivery, etc

Guy Vernat, Foundation Merieux USA

I-Lab: Connecting clinical Labs to Infectious Diseases Surveillance Systems in West Africa

Over a period of 12 months, I-LAb mapped all the lab sites in Senegal using WHO lab assessment tool, including location as well as devices available and types of diagnostics possible

-DHIS2 used to share reporting and analysis of data, but was not used for labs. I-Lab project implemented DHIS2 for electronic disease surveillance

-captured data re: lab locations, lab capabilities and created country’s first “Lab Book”

ACTION: any interest from AmosTaxi to connect?

Benjamin Winters, Akros

Zambia based company focused on developing surveillance processes and other health interventions and DHIS2 trainings

Worked closely on Macepa project with PATH and used “reactive case detection” with DHIS2 to map areas

Process management of campaigns (vaccinations, IRS, etc.). Looking for better M&E models, process improvement

Indoor Residual Spray- found more accurate way to measure/calculate coverage. ‘Denominator challenge’--Akros & Ona worked together to  identify structures and guide field teams. mSpray takes maps, puts them into a device, and helps teams know where they need to go to spray and record data. Creates breadcrumb dataset. Using mSpray caused teams to find more structures and spray more. This instance was a success, but what we want is overall govt improved performance

-learned that the mSpray concept had applicability across many types of campaigns

Call to action: if we want our solutions to stick over time, more investment into the data culture that funds these innovations is needed

Kelsey (?) standing in for Alain Labrique

Geo-spacial Innovations to Improve Equity and Measure True Coverage

JaVitA Study Area- households are mapped, women have ID number.

• Community maps are usually simple, hand-drawn, and have unclear boundaries.
• Gaps and duplications in frontline worker coverage became apparent when geospatially mapped
• How do you attribute different levels of effort across HCWs- so they can give credit when due, and evenly distribute work (see how far they walk, how fast, and how long they stay at each place)
• Working to identify denominators more effectively (how many people or structures exist in a certain area?)

-Tech has advanced faster than government structures have to support them.*this issue was raised in earlier governance session today as well*

-How can we leverage the private sector?

-What will next couple of years bring?

Impact of Digital Health Interventions on Data Use and Health Outcomes - Immunization Focus

Attendee: Vidya

Session objectives

ToCs state that there is an impact of digital health interventions on data use and health -- save/improve quality of life, save time, save money BUT the actual evidence lacks the rigor/stds to actually state their usefulness as fact

Today's session presents some of that better research

Garrett Mehl - present 2 reviews

Have contacted him for the presentation deck of his two reviews because actual talk was super choppy, hard to follow and full of tech issues

Jessica Shearer -- Dir, Health Systems Analytics and Technical Lead on IDEA

IDEA report now available on www.findyourfinding.org

They conducted a realist review of what works to improve the use of routine data in immunization decision-making with evidence from peer and grey literature of work done in LMICs. (Realist reviews look at whether something worked, why they worked and dig into mechanisms/contexts where they worked in order to scale up) This analysis resulted in an evidence gap map -- mostly greys (grey lit) and few blues (peer reviewed). They categorized the evidence as "high certainty of evidence, some certainty of evidence, low certainty" but a major limitation is that they did not use the same set of outcome or impact measures for all the evidence they reviewed, instead using the publication's own interpretation of success in their classification.

Takeaways:

• Interconnected strategies get better results --  interventions focused on skills and capabilities along with technology far more effective than tech interventions alone
• Systemizing data use leads to long-term success -- get the data use piece down (how are they using it, when, what is the process it's going into, etc) along with the actual tech deployment
• Difference between LMIS and HMIS usefulness --- strong evidence from review that LMIS systems have improved data availability and use and low certainty of evidence of HMIS systems doing the same, particularly on data use front
• Assumption going in had been data quality will lead to better data use but the review found the opposite --- that data use leads to better data quality.
• Across the breadth of studies reviewed, the HF was forgotten completely in the data use side -- only focus for HF was on improving data quality but not about establishing feedback loops or demonstrating value add of the data being collected. This is a missed opportunity they wanted to highlight.

Nargis Rahimi, Shifo Foundation

Conducted scoping review, did not put a time period barrier, looked at all peer-reviewed publications looking at what factors contribute to poor data versus good data, and facilitations of what interventions work to improve quality.

Identified 1000+ articles on the topic but only 25 made the cut!

Takeaways:

• A big contributor to poor data quality is when a promising pilot ends up scaling too quickly or prematurely. Evaluate critically, remember that your pilot has to be sustained by existing health system structure -- but somehow every project they studied got this wrong.
• --> vidya side note: I've been reflecting on this a lot given our experience with SELV -- going from 4 provinces to 11 in less than a year and with same # of staff to support…
• Each implementer has a responsibility to better inform our own work and action based on what we know -- not the just implementation of a new project but the overall strategy, too. There doesn't seem to be that self-reflection in the reviews studied -- same implementing partners did X in one country, wrote up their learnings and 3 years later, did the same in Y country and nothing had changed in approach or strategy -- why?
• Add an evaluation component from the get-go to your project -- ensure following research protocols and collect evidence accordingly -- that is how the poor evidence base in this area will be improved

Country Experience with Strategy and Governance (10:30 - 11:45am)

Attendee: Rebecca

Key takeaways

• Malawi: there is fragmentation of mhealth interventions in countries. There is a need for streamlining interventions, and use applications that span across program areas. Standards are needed for hardware, software, hosting, training, etc.
• National assessment-->draft of plan/roadmap-->governance strategy-->funding
• Advice: strategies need to speak for health sector, not just the technology
• Ethiopia wants to focus on using standards and interoperability; specific TWG for this
• Kenya, TZ, and Ethiopia presented on their digital health plans/roadmaps and priorities
• What is a strategy vs Policy? Policies don’t explain the ‘how or why’, it’s just a law. Strategies contain specific steps, which need to be budgeted for and measured (indicators). ‘Vision without resources is hallucination’’.
• How do you develop policy to support a strategy? Countries need support on this. They request the global community to create a blueprint for this
• When moving toward digitization, it is important to provide tools to improve culture of data use. If they aren’t used to using performance data to make decisions, they need to be taught to do so. Can’t assume digitization alone will solve problems. Data must be appreciated.
• Interoperability: Kenya has enterprise architecture, provides interoperability. They have a mobile service centers for people who don’t have good mobile connectivity. Kenya has a data depository/dashboard (with support from WHO).
• What are top 2 needs for capacity building in your govt?: Malawi- use of data at the facility level, TZ-health informatics techniques for graduates at various levels, they need people to train the service providers to generate and use information; infrastructure/ support for tele-medicine; Ethiopia: understanding of global standards/interoperability
• Call to action: mHealth is here to stay, we need to move forward with the tide and improve our underlying systems to embrace digital health innovation