OpenLMIS Malawi TO1
Global 3.0 Validation Testing Results Report

Purpose

OpenLMIS global version 3.0 was released on February 28, 2017. Malawi will be the first country to implement the newly released version 3.0 to replace the current logistics management information system, Supply Chain Manager (SCMgr). As such, testing with a group of core users (see appendix A) was conducted on March 2, 2017 to validate the core features of global OpenLMIS 3.0 against the features and processes currently supported by Supply Chain Manager. Additionally, the testing aimed to identify and document any gaps in OpenLMIS 3.0's features compared to SCMgr, bugs in OpenLMIS 3.0, and/or specific changes or customizations required for Malawi.

Application Overview

The application tested in this activity was the February 28, 2017 release of OpenLMIS global version 3.0. The 3.0 version of OpenLMIS global includes an updated micro-service system architecture including requisitions, authentication, reference data, fulfillment, and notification services, with the associated reference user interface (UI). For further information on OpenLMIS global version 3.0, please see the full release notes located on the OpenLMIS wiki.

Testing Scope

Global validation testing focused on testing the requisition process.

In Scope

Based on the processes currently supported by SCMgr, the testing scenarios cover the following functionalities:

Creating and submitting a regular type requisition
1. Adding comments
2. Check validations
Authorizing a requisition
Approving a requisition
Rejecting a requisition
Converting a requisition to an order
View a list of requisitions
Reference UI

Out of Scope

- Reports
- Notifications
- Configuration
- Malawi-specific reference data
- Malawi-specific UI

Test Environment & Tools

The testing session took place in the Chemonics office and was conducted using the global UAT instance configured with demo reference data and default settings. No Malawi-specific configuration, set up, or UI were completed for this testing session. Testers used their own computers or were provided computers by the Chemonics team. Testing instructions with credentials, testing steps, and space to record results were printed and provided to the testers.

Tests Performed

Please see the testing instructions in appendix B for the specific tests that were performed.

Results

Individual testing steps were completed as part of 7 scenarios.

Test cases planned vs. executed and passed vs. failed

Test cases planned	Test cases executed	Pass	Failed*
86	86	77	9

*Includes all testing steps that were identified as failed by any participant, excluding than configuration issues

Defects identified and their Status & Severity

	Blocker	Critical	Major	Minor	Trivial	TOTAL
Closed	1	0	2	1	2	6
Open	0	0	1 OLMIS-2068 - Getting issue details... STATUS	0	1	2
TOTAL	1	0	3	1	3	8

Defects Distribution

	Create requisition	Authorization/ Approval	Orders	Admin/UI
Blocker	1	0	0	0
Critical	0	0	0	0
Major	3	0	0	0
Minor	0	0	0	1
Trivial	0	3	0	0
TOTAL	4	3	0	1

Lessons Learned

There were insufficient logins and available requisition periods for the number of users included in the testing, preventing users from initially completing some testing steps (such as submitting or authorizing a requisition) since another user, signed in with the same credentials, had already performed the same action. This issue was resolved by having users start testing sequentially, to ensure that each user was working on a different period that would remain available to them.
Login issues were prevalent due to the very fast auto-logout setting. Users were required to sign in repeatedly in order to complete the testing. This could not be changed at the time of testing, but was noted for future testing.
Though the global version was purposefully not configured with Malawi-specific information, and this was explained to the testers, it did still create confusion and cause questions or issues for users that were configuration-related.

Additional Issues Reported & Recommendations

In addition to recording issues with the specific features or functions explored in the testing scenarios, users provide additional feedback on general appearance and ease of use, overall issues they ran into

UI Issues

Scrolling/Screen-resolution responsiveness
- The positioning and visibility of the scroll bar was problematic for many of the users. Most users had to scroll down within the browser window to see the bottom of the requisition, so the scroll bar was not easily visible at all times.
- The fact that the scroll bar is not always visible presented issues. Several users had issues knowing that the scroll bar was available and getting it to appear.
- In general, users did not like the horizontal scrolling, and suggested reducing the number of columns (which could be done via configuration), as well as making them narrower so horizontal scrolling would be reduced.
Profile and logout options
- Users reported that the current appearance of the profile and logout options (hyperlinks) made it hard hard to distinguish that they were two separate options. Additionally, the style, size and location made them difficult to locate on the screen. OLMIS-2066 - Getting issue details... STATUS
  - Suggestions for improvement included making them look like buttons rather than links, making them larger, and or placing/spacing them differently to make them more distinct.
Skip checkboxes
- There are several options for the skip product(s) feature: individual boxes, all, and none. The display, however, was confusing for users. There is a 'skip' label in the table headings on the requisition as well as immediately above it with the all and none options. The check boxes are included in each row, but look somewhat separate from the skip label. When trying to skip a product, users often first noticed the all and none options, but were confused because the ability to skip an individual product was not immediately obvious. The issue with the skip product function versus the skip requisition, described below, compounded this issue. While this problem would, in large part, be addressed with proper training it was clear that the way to skip products is not obvious based on the current appearance.

Miscellaneous Issues

Skip functionality
- The skip feature was a point of confusion for several users. Difficulty was reported distinguishing between skipping individual products OLMIS-2082 - Getting issue details... STATUS (with the checkboxes) and the skip button for skipping a requisition. The button is more prominent and many users first tried to skip the period when trying to skip a product.
- The skip button to skip a requisition was available even for users not authorized to skip a period OLMIS-2067 - Getting issue details... STATUS
User profile
- When accessed, the profile had labels for user related information, but did not contain any of the user's information. OLMIS-2060 - Getting issue details... STATUS
Field required box did not disappear
- One user received the "field required" error for a field, which remained on the screen (not attached to the field) even after entering a value.
Total requisition cost
- Most fields on the requisition have an info button that will provide the user with information on what the field is, what it means, and/or how it should be calculated. The total cost of the requisition does not, and the calculation it is using was not clear and confusing to some users. Many users might confuse the dispensing unit as the pack size, and think the calculation was not working properly. OLMIS-2059 - Getting issue details... STATUS

Additional Feature/Customization Suggestions

Add help button/site documentation for users.
Approval step should have the requested quantity default as the "approved" quantity so the approver only needs to change quantities, not type them all in again. OLMIS-1989 - Getting issue details... STATUS
The ability to edit a requisition after submission.
The capability to change order quantity calculations to include seasonality for some commodities.

Appendix A. Testing Participants

#	Name	Title	Organization	E-mail
1	Austine Omionu	M&E Advisor	HTSS Secondee (PSM)	AOmiunu@ghsc-psm.org
2	Elias Mwalabu	Program FP - Malaria	PSM	EMwalabu@ghsc-psm.org
3	Joyce Wachepa	Program FP - FP	PSM	JWachepa@ghsc-psm.org
4	Douglas Khumalo	E-LMIS Manager	PSM	DKhumalo@ghsc-psm.org
5	Joseph Mtenje	OpenLMIS Officer	PSM	jmtenje@ghsc-psm.org
6	Charles Chimenya	Pharmacist	HTSS	charleschimenya@yahoo.com
7	Maganizo Monawe	Team Lead Kuunika Project	CMED	mmonawe@gmail.com
8	Chikhulupiliro Chimwaza	Pharmacist	Queen Elizabeth Central Hospital
9	Elizabeth Mkandawile	Program - Malaria	MOH
10	James Kumwenda	District Pharmacist	Mangochi District Medical Officer
11	Andrea Fletcher	Lead Data strategist	Cooper Smith	andrea@coopersmith.org

Appendix B. Global OpenLMIS v3.0 Testing Instructions

Test Scope

The scope of this preliminary acceptance testing is to validate the core functionalities of OpenLMIS global version 3 against the key features and processes currently supported by Supply Chain Manager (SCMgr). As such, the primary focus of this testing is not to find bugs in the global version (a separate UAT was conducted for this purpose already). Any bugs found, however, should be carefully recorded to provide additional feedback to the global development team to ensure those bugs are fixed for the Malawi instance. Given the processes currently supported by SCMgr, the testing scenarios cover the following functionalities:

Creating and submitting a regular type requisition
1. Adding comments
2. Check validations
Authorizing a requisition
Approving a requisition
1. Printing a requisition
Rejecting a requisition
Converting a requisition to an order
View a list of requisitions

Test Roles

Role Name	Login Information	Rights	Home Facility
Pharmacy tech.	user name: srmanager1 password: password	Create and submit R&RView R&R	Nandumbo Health Center
District pharmacist	user name: smanager1 password: password	Authorize R&R View R&R	Nandumbo Health Center
DTC/DHO	user name: psupervisor password: password	Approve R&RView R&R
Warehouse Manager	user name: wclerk1 password: password	Convert requisitions to order

Scenario 1: Accessing the system and logging in

1.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser	Welcome screen appears and there is a modal asking for login credentials
2		Type username and password for the pharmacy tech and click 'Sign in' or press Return/Enter Key on the keyboard	Directed to the home screen, displaying a navigation bar with "Home" and "Requisitions". You should not see "message keys". Everything should be in English.
3		Click the logout button	The system logs you out and displays the log in screen

Scenario 2: Creating and submitting a regular type requisition for the Family Planning program

2.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser	Welcome screen appears and there is a modal asking for login credentials
2		Type username (srmanager1) and password (password) for the pharmacy tech for Nandumbo clinic. Click 'Sign in' or press Return/Enter Key on the keyboard	Directed to the home screen, displaying a navigation bar with "Home" and "Requisitions". User name is displayed at the top right corner of the screen.
3		Select the "Requisitions" menu header on the top navigation bar using your cursor	A drop down menu should appear with the following options: Create / Authorize View
4		Select the "Create / Authorize" option	The "Initiate report and requisition" page appears with the following options: Facility Type radio buttons for 'My facility' or 'My supervised facilities' drop down for facility name drop down for program Type radio buttons for regular or emergency Facility Type options are pre-selected since this user isn't authorized for other programs or facilities. However, user can select either Regular or Emergency.
5		Select the 'Regular' Type radio button and press 'Search'	A table appears with a list of periods.
6		Select the "proceed" button for the available period (top row)	You are brought to the R&R form for data entry. There should be a header at the top with the accurate details of the Requisition type, Facility, Operated By, Region, and Period. At the bottom left of the table, it should indicate the number of products in the requisition.
7		Insert in data for all available columns and review column definitions.	There should be an "i" next to all column headers. The pop out will display a definition or additional instructions.
8		Enter a value in beginning balance, then attempt to skip that product (row/lineItem).	Check box for skipping should be disabled if there are any values in the row.
9		If there are values in the cells, clear the values for a product (row/LineItem) and check the skip checkbox	Check should display and row should gray out. The row should no longer allow for editing.
10		Select the "Losses and adjustments" for a product and enter in a positive and negative adjustment. Select close when completed.	Modal should appear to input in loss and adjustment reasons. Upon entering multiple adjustments, you should see a summary of the total adjustment (positive/negative)
11		Enter in information for an entire row.	Depending on the amounts, you will run into data validations to ensure appropriate data entry.
12		Enter a quantity for Total Stock out Day.	The adjusted consumption column should update based on number of stock out days.
13		Attempt to enter negative stock out days.	You should receive an error.
14		Enter a larger quantity for " Total Consumed" then the summation of Received + Beginning Balance quantities.	You should receive an error.
15		Enter in a Comment at the top of the page	You should be able to enter in text. The comment will save automatically. You can edit the comment until the submit button is selected.
16		Review total requisition cost	Costs should update when quantities are entered into the Requested Quantity column.
17		Submit the requisition	If there are any errors, you should not be able to submit. If the errors are on the second page, the number should turn red to indicate where the error is on the requisition. Confirmation message is presented.

Scenario 3: Authorizing a requisition

3.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser	Welcome screen appears and there is a modal asking for login credentials
2		Type username "smanager2" and password "password" for the pharmacist at Comfort Health Clinic facility and click 'Sign in' or press Return/Enter Key on the keyboard	Directed to the home screen, displaying a navigation bar with "Home" and "Requisitions". User name is displayed at the top right corner of the screen.
3		Select the "Requisitions" menu header on the top navigation bar using your cursor	A drop down menu should appear with the following options: Create / Authorize View
4		Select the "Create / Authorize" option	The "Initiate report and requisition" page appears with the following options: Facility Type radio buttons for 'My Facility' or 'My Facilities' drop down for facility name drop down for program Type radio buttons for regular or emergency
5		Select the program drop down and select 'Family Planning'. Select Regular type.	Since the store manager does not supervise other facilities, the option of selecting "my facilities" should be unavailable to you. A Program drop down with two options should be available
6		Select "Search"	A table appears with a list of upcoming periods and a 'proceed' button on the right to a requisition with the status of "submitted"
7		Select "Proceed"	You are brought to the R&R form for data entry. There should be a header at the top with the accurate details of the Requisition type, Facility, Operated By, Region, and Period. All cells should be editable. Buttons for delete and Authorize on the right. Button for 'sync to server' on the left. You can unskip and enter data. There are products on each page. Pages are displayed on the bottom right.
8		Add a comment	Ability to enter text. Automatic saving behavior (messages pop up telling you that your changes are saved.
9		Select "Authorize"	Confirmation message and authorization complete.

Scenario 4: Approving a requisition

4.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser	Welcome screen appears and there is a modal asking for login credentials
2		Type username and password for the Program Supervisor and click 'Sign in' or press Return/Enter Key on the keyboard	Home screen with Requisitions on the navigation bar.
3		Click on Approve menu	A list (table view) of submitted R&R's appear on the screen.
4		Click on "view requisition" for the Kankao Health Facility R&R	The R&R form appear with two editable columns "approved quantity" and "remarks"for all the product line items
5		Enter in quantities in the approved quantity and remarks in the remarks.
6		Select the Approve button	A dialogue box appears to ask the user to confirmation
7		Press OK button to confirm	The dialog box disappears and a list of R&R to be approved appears and R&R approved successfully message appears. The dialog box together with the approved R&R disappear, while the list of unapproved R&R together with the message "R&R approved successfully" re – appears
8		Add Comment	Ability to enter text. Automatic saving behavior (messages pop up telling you that your changes are saved.

Scenario 5: Rejecting a requisition

5.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser. Login as Program Supervisor.	Welcome screen appears and there is a modal asking for login credentials
2		Navigate to Requisitions and select "Approve"	List of requisition waiting for approval
3		Select a requisition.	When hovering over a requisition, the row should highlight.
4		Review the requisition	You should not be able to modify any quantities or skip a product.
5		Select the "Add a comment" and enter a comment.	Text box should display and allow for the user to enter text. The requisition should automatically save. You should also see a 'remove comment' button.
6		Reject the requisition	Ability to enter text. Automatic saving behavior (messages pop up telling you that your changes are saved.

Scenario 6: Converting a requisition to an order

6.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser. Login as a warehouse clerk.	Welcome screen appears and there is a modal asking for login credentials
2		Navigate to the Requisitions menu and select "convert to order"	A list of approved requisitions waiting to be converted to order.
3		Select an approved requisition and select the 'convert to order' button	The system produces a confirmation message.
4		Confirm the action.	The requisition is removed from the list of requisitions ready for conversion.

Scenario 7: View a list of requisitions

7.	Tester	Steps	Expected Results	Pass/Fail	Actual Result (if failed)
1		Enter http://uat.openlmis.org/ into a chrome or firefox browser. Login as Program Supervisor.	Welcome screen appears and there is a modal asking for login credentials. Home screen is visible after credentials are accepted.
2		Navigate to the Requisitions menu and select "view"	A vertical search bar is available to search for requisitions.
3		Select Kankao facility from the drop down. Then select "Search".	A list of requisitions for Kankao should be displayed in a table format.

Malawi Version 3 Testing Results