2019-01-28 Data Ingestion Walkthrough

Date

28 Jan 2019 

Attendees / Stakeholders

• Clay Crosby

• Swetha Srinath

• Timoteo Chaluco

• Felimone Amone Junior

• Dércio Duvane

• Christine Lenihan

• Vidya Sampath

• Ben Leibert

• Wambere (Unlicensed)
  
  

Notes

• Users who receive an error email regarding a data submission should check to ensure that the following are correct:
  1. The name of the attachment.
  2. The subject-line of the email they sent.
  3. That the names of provinces and districts within their file matches those which NiFi expects.
  
  
• There hasn't been complete consistency regarding the format of HSS related data files, and Timoteo currently has to edit them slightly by hand. We therefore agreed as a group to use the file-format that NiFi currently supports. Namely: The column headers appear on row 1, the actual data appears thereafter, beginning on row 2, and the "Grand Total" row is excluded from the data file emailed to the backend.
  
  
• In order to promote and use of the above format, we'll begin using a template-file for the HSS funds. Its headers should be in Portuguese, and Timoteo will therefore provide it. 
  
  
• It would be somewhat difficult to have NiFi log the name of unexpected provinces or districts that it encounters. To help get around this, will add a tab to each template-file which includes the expected names of provinces and districts.

Action Items

• Timoteo Chaluco (Unlicensed) will send the team an HSS-funds template file which includes the names of the column-headers in Portuguese.
• Wambere (Unlicensed) will investigate why NiFi simultaneously sends an error as well as success email when HSS data is submitted.
• Clay Crosby (Unlicensed) will add the names of provinces and districts which NiFi recognizes as valid to a new tab within the Excel template files.