OpenLMIS global version 3.0 was released on February 28, 2017. Malawi will be the first country to implement the newly released version 3.0 to replace the current logistics management information system, Supply Chain Manager (SCMgr). As such, testing with a group of core users (see appendix A) was conducted on March 2, 2017 to validate the core features of global OpenLMIS 3.0 against the features and processes currently supported by Supply Chain Manager. Additionally, the testing aimed to identify and document any gaps in OpenLMIS 3.0's features compared to SCMgr, bugs in OpenLMIS 3.0, and/or specific changes or customizations required for Malawi.

The application tested in this activity was the February 28, 2017 release of OpenLMIS global version 3.0. The 3.0 version of OpenLMIS global includes an updated micro-service system architecture including requisitions, authentication, reference data, fulfillment, and notification services, with the associated reference user interface (UI). For further information on OpenLMIS global version 3.0, please see the full release notes located on the OpenLMIS wiki.

Global validation testing focused on testing the requisition process.

Based on the processes currently supported by SCMgr, the testing scenarios cover the following functionalities:

• Creating and submitting a regular type requisition

  1. Adding comments

  2. Check validations

• Authorizing a requisition

• Approving a requisition

• Rejecting a requisition

• Converting a requisition to an order

• View a list of requisitions

• Reference UI

•  

  • Reports

  • Notifications

  • Configuration

  • Malawi-specific reference data

  • Malawi-specific UI

The testing session took place in the Chemonics office and was conducted using the global UAT instance configured with demo reference data and default settings. No Malawi-specific configuration, set up, or UI were completed for this testing session. Testers used their own computers or were provided computers by the Chemonics team. Testing instructions with credentials, testing steps, and space to record results were printed and provided to the testers.

Please see the testing instructions in appendix B for the specific tests that were performed.

Individual testing steps were completed as part of 7 scenarios.

*Includes all testing steps that were identified as failed by any participant, excluding than configuration issues

• There were insufficient logins and available requisition periods for the number of users included in the testing, preventing users from initially completing some testing steps (such as submitting or authorizing a requisition) since another user, signed in with the same credentials, had already performed the same action. This issue was resolved by having users start testing sequentially, to ensure that each user was working on a different period that would remain available to them.

• Login issues were prevalent due to the very fast auto-logout setting. Users were required to sign in repeatedly in order to complete the testing. This could not be changed at the time of testing, but was noted for future testing.

• Though the global version was purposefully not configured with Malawi-specific information, and this was explained to the testers, it did still create confusion and cause questions or issues for users that were configuration-related.

In addition to recording issues with the specific features or functions explored in the testing scenarios, users provide additional feedback on general appearance and ease of use, overall issues they ran into

• Scrolling/Screen-resolution responsiveness

  • The positioning and visibility of the scroll bar was problematic for many of the users. Most users had to scroll down within the browser window to see the bottom of the requisition, so the scroll bar was not easily visible at all times.

  • The fact that the scroll bar is not always visible presented issues. Several users had issues knowing that the scroll bar was available and getting it to appear.

  • In general, users did not like the horizontal scrolling, and suggested reducing the number of columns (which could be done via configuration), as well as making them narrower so horizontal scrolling would be reduced.

• Profile and logout options

  • Users reported that the current appearance of the profile and logout options (hyperlinks) made it hard hard to distinguish that they were two separate options. Additionally, the style, size and location made them difficult to locate on the screen.https://openlmis.atlassian.net/browse/OLMIS-2066

    • Suggestions for improvement included making them look like buttons rather than links, making them larger, and or placing/spacing them differently to make them more distinct.

• Skip checkboxes

  • There are several options for the skip product(s) feature: individual boxes, all, and none. The display, however, was confusing for users. There is a 'skip' label in the table headings on the requisition as well as immediately above it with the all and none options. The check boxes are included in each row, but look somewhat separate from the skip label. When trying to skip a product, users often first noticed the all and none options, but were confused because the ability to skip an individual product was not immediately obvious. The issue with the skip product function versus the skip requisition, described below, compounded this issue. While this problem would, in large part, be addressed with proper training it was clear that the way to skip products is not obvious based on the current appearance.

• Skip functionality

  • The skip feature was a point of confusion for several users. Difficulty was reported distinguishing between skipping individual products

    (with the checkboxes) and the skip button for skipping a requisition. The button is more prominent and many users first tried to skip the period when trying to skip a product.

  • The skip button to skip a requisition was available even for users not authorized to skip a period https://openlmis.atlassian.net/browse/OLMIS-2067

• User profile

  • When accessed, the profile had labels for user related information, but did not contain any of the user's information. https://openlmis.atlassian.net/browse/OLMIS-2060

• Field required box did not disappear

  • One user received the "field required" error for a field, which remained on the screen (not attached to the field) even after entering a value.

• Total requisition cost

  • Most fields on the requisition have an info button that will provide the user with information on what the field is, what it means, and/or how it should be calculated. The total cost of the requisition does not, and the calculation it is using was not clear and confusing to some users. Many users might confuse the dispensing unit as the pack size, and think the calculation was not working properly.https://openlmis.atlassian.net/browse/OLMIS-2059

• Add help button/site documentation for users.

• Approval step should have the requested quantity default as the "approved" quantity so the approver only needs to change quantities, not type them all in again. https://openlmis.atlassian.net/browse/OLMIS-1989

• The ability to edit a requisition after submission.

• The capability to change order quantity calculations to include seasonality for some commodities.

The scope of this preliminary acceptance testing is to validate the core functionalities of OpenLMIS global version 3 against the key features and processes currently supported by Supply Chain Manager (SCMgr). As such, the primary focus of this testing is not to find bugs in the global version (a separate UAT was conducted for this purpose already). Any bugs found, however, should be carefully recorded to provide additional feedback to the global development team to ensure those bugs are fixed for the Malawi instance. Given the processes currently supported by SCMgr, the testing scenarios cover the following functionalities:

1. Creating and submitting a regular type requisition

   1. Adding comments

   2. Check validations

2. Authorizing a requisition

3. Approving a requisition

   1. Printing a requisition

4. Rejecting a requisition

5. Converting a requisition to an order

6. View a list of requisitions

Scenario 1: Accessing the system and logging in

Scenario 2: Creating and submitting a regular type requisition for the Family Planning program