2018-05-16 Weekly Check-in

Date

16 May 2018

Attendees / Technical Stakeholders

• Craig Appl (Unlicensed)
• Clay Crosby (Unlicensed)
• Antonate Maritim (Unlicensed)
• Wambere (Unlicensed)
• Felimone Amone Junior
• Timoteo Chaluco (Unlicensed)
• Dércio Duvane
• Christine Lenihan

• Ben Leibert

Discussion items

Previous Action Items

* The fact that Ona will host the system somewhat obviates the following action items from last week's meeting:

• "Clay Crosby will have the team send Ben Leibert their public .ssh keys."
• "Clay Crosby will recommend server hardware requirements."
• "Craig Appl will send Ben their Ansible playbooks."

Given that Ona will host the system, it's unclear whether it makes sense to have Ben create and configure the EC2 instances along with the software that will run atop them. Clay Crosby (Unlicensed) will touch base with Peter within the next several days to discuss. Because the MoH may ultimately want to take ownership of the system, it would be ideal to have the software run on one or more dedicated EC2 instances. An incidental benefit of this approach would be that it allow Ben Leibert to access the system even without giving him access to data associated with Ona's other clients.

* Dércio Duvane requested a meeting with the MoH, during which he'll ask them about the domain we should use. They've been slow to respond, but Dercio plans to follow up with them.

Implementation

• Because filters in Superset can only be associated with a single table, it's ideal if everything else in a given dashboard can also be associated with just one table. It's important to understand what the rows of this table represent - and what granularity they have. Each row will likely represent a "facility-date pair" and have one column per indicator. So, for SELV's data, there will be a "BCG_Given" column, a "Total_Kids_Eligible" column, etc.
  
  
• In SELV, the facility-type "depo" refers to a provincial warehouse.
  
  
• We'll want to host our Ansible playbooks/inventories in a private repository. Ben Leibert will check github's non-profit pricing and subsequently propose who should host the repository.
  
  

Data Sources 

• The team was able to get Nifi to consume data sent via an email attachment. Although it's also possible to have it ingest email bodies, doing so would require the creation of a special parser. The parser would require the email's body to be highly structured - like an HTML table, for example. Because this would represent a departure from the freeform manner in which people are accustomed to writing email, it would increase the likelihood of user dissatisfaction and error. In contrast, people are accustomed to attaching files to email. Because relying on an attachment seems better and easier, it's likely the approach we should take.
  
  We'll have Nifi do quality-control on the attachments associated with an email. In can loop through all of them looking for the relevant one. If it doesn't find one, or if the apparently relevant attachment isn't properly formatted, NiFi will automatically send the relevant user an email notifying them of the problem.
  
  
• The e-SISTAFE file is in Portuguese and, due to the acronyms within it, difficult to use with Google Translate. Additionally, we need clarification about what each row means. Dércio Duvane is planning to meet with the folks responsible for e-SISTAFE. Afterward, he'll provide an English translation of the file and describe what its rows represent.
  
  
• DHIS-2 is in Portuguese and we need guidance regarding their API. Craig Appl (Unlicensed) has experience with its API and will help.
• SELV doesn't contain any views - everything is a table. The granularity of data it collects is monthly, per "period."
  
  Although the SELV Dashboard should perform its ingestion-process daily, it only needs to expose data with a monthly level of granularity.
  
  The MoH's reporting periods are on a "monthly" schedule, but their start and end dates don't correspond to first and last dates of the Gregorian month. Instead, they start on the 21st day of each month, which is how SELV is configured. This corresponds to a MoH standard applicable across all systems. (NOTE: Because NexLeaf is somewhat of a black box to us, we should probably confirm that they conform to this standard as well. It's not obvious that they do so based exclusively on the image returned by https://coldtrace.org/plot/coldtrace/lloyd/Tete/2018/data/)
  
  
• The Mozambique team is trying to include and empower the MoH and EPI team. Dércio Duvane and Timoteo Chaluco (Unlicensed) have casually met with them regarding the trimestral reports. The EPI team says that submitting the data via email is okay, but Dercio and Timoteo plan to follow up with them and confirm. For now, we should plan on trimestrial reports being submitted as email attachments which take the form of CSV files. The precise makeup of the CSV files, though, hasn't yet been determined.
  
  

Travel 

The team thinks it would be ideal for Ben to be on hand for about two weeks, beginning next week. Additionally, to play it safe, Ben will plan to be in-country for an additional "flex" week. If there's no need to be in the office during the third week, Ben will use it to explore the country. Otherwise, he'll keep working. Ben will try to get flexible tickets, but otherwise plan to arrive at Ona's office the morning of 5/22 or 5/23 and to depart thee weeks thereafter.

Action items

• Clay Crosby (Unlicensed) will touch base with Peter to decide on the degree to which Ben should focus on initial system/software setup.
• Craig Appl (Unlicensed) will provide guidance about the API provided by DHIS-2.
• Dércio Duvane will meet with the folks responsible for e-SISTAFE.
  • Meeting on Monday and Tuesday, May 21st/22nd
• Dércio Duvane will provide an English translation of the e-SISTAFE spreadsheet as well as describe the meaning of its rows.
• Dércio Duvane and/or Timoteo Chaluco (Unlicensed) will coordinate with the EPI team to determine which content should comprise the CSV data in the trimestral reports.
• Ben Leibert will check github's non-profit pricing and subsequently propose who should host the project's private repo.