The 3.8 Regression and Release Candidate Test Plan
The Release Candidate Test Plan
When all 3.8 tickets have been completed, we can begin testing, and we will follow the test plan detailed below.
Before we start the Release Candidate testing:
If there are any blocker or critical bugs when we start Sprint 139, they must be fixed and tested before the RC testing can begin;
If there are any open tickets related to the features in the 3.8 release, they must be completed and marked as Done before any RC testing begins;
Regular regression testing has been completed in previous sprints before we start the release candidate testing.
Starting the Release Candidate testing:
If there are no open blocker or critical bugs, then move to the release candidate's deployment and testing:
RC1 (Three full days of testing, and 3 days for bug fixes and bug fix testing) for each release candidate as needed;
The testing will be done in the following environments:
Team Mind the Parrot's UAT environment: https://uat.openlmis.org;
Performance testing: (https://perftest.openlmis.org).
How to enter bugs: Testing Process & Test Plans#EnterDefectsduringsprinttesting;
Prioritization of bugs: http://docs.openlmis.org/en/latest/contribute/contributionGuide.html#reporting-bugs.
Test Phase | Components/Features | Assigned in Test Cycle | Dates | QA lead responsibilities |
|---|---|---|---|---|
RC 1 |
| 3 full days (as needed) | Team Mind the Parrot:
| |
Bug triage | Bug triage for RC1 | Bug fixes test cycle for a given Release Candidate, e.g. 3.8 RC1 Bug fixes | 3 full days (as needed) |
|
Suggested schedule, assuming we start testing on December 5:
Week One:
Thu Dec 05 | Fri Dec 06 | Mon Dec 09 | Tue Dec 10 | Wed Dec 11 |
|---|---|---|---|---|
|
|
|
|
|
Week Two:
Thu Dec 12 | Fri Dec 13 | Mon Dec 16 | Tue Dec 17 | Wed Dec 18 |
|---|---|---|---|---|
|
|
|
|
|
Week Three:
Thu Dec 19 | Fri Dec 20 | Mon Dec 23 | Tue Dec 24 | Wed Dec 25 |
|---|---|---|---|---|
|
|
|
|
|
Week Four:
Thu Dec 26 | Fri Dec 27 | Mon Dec 30 | Tue Dec 31 | Wed Jan 01 |
|---|---|---|---|---|
|
|
|
|
|
Roles & Responsibilities
QA Team Leads
@Joanna Bebak (Deactivated) will be the QA lead for the Mind the Parrot team.
Owner | Responsibilities | Questions/Tasks before release begins |
|---|---|---|
@Joanna Bebak (Deactivated) |
|
|
Team leads:
|
|
|
Team responsibilities |
|
|
The Bug Triage team
Members | Responsibilities | Questions |
|---|---|---|
@Wesley Brown @Sebastian Brudziński @Joanna Bebak (Deactivated) @Joanna Szymańska (Deactivated) @Douglas Khumalo (Unlicensed) (or other) from Malawi |
|
|
Communication on the Test Plan and the daily testing status:
What needs to be communicated daily?
The test cycle execution status (including the test cycle's name and % of completion);
# of test cases that were executed, passed and failed;
@Joanna Bebak (Deactivated) will post any test cases or items that need attention and review by the end of her day on the QA Slack channel;
@Joanna Bebak (Deactivated) will communicate with the Malawi team (via the Malawi Slack channel) and notify them about the release candidate testing:
The communication of the scheduled start date before the release candidate testing begins;
The communication of when we start the release candidate testing (Malawi has one week to test);
The Malawi team is included in the daily bug triage meetings.
The best time of day (for each team to communicate the morning status and the end of day status & share blockers):
The beginning of the day: Post what we are doing today;
The end of day: Post the status of what we have done, and anything pending.
Environments and the demo data for testing
Environments: uat.openlmis.org.
Refer to the demo data readme for more details about user permissions: https://github.com/OpenLMIS/openlmis-referencedata/blob/master/src/main/resources/db/demo-data/README.md.
ONLY test with the users "admin" or "administrator" when executing test cases related to administrative activities.
Test Data
Component | Username | Program | Concerns |
|---|---|---|---|
Requisition
| srmanager1, smanager1, psupervisor, wclerk1
srmanager2, smanager2, psupervisor, wclerk1
srmanager4 (for the second approval), smanager4, dsrmanager, psupervisor
chaz (for the second approval)
administrator (testing requisition template updates/changes or program settings changes) | Family Planning Essential Meds ARV
Essential Meds and Family Planning
ARV
Family Planning Essential Meds EPI ARV |
|
Stock Management | srmanager2
divo1, rivo | Family Planning, Essential Meds
EPI |
|
Fulfillment | vsrmanager1 vsrmanager2 divo1 divo2 rivo vwclerk1 wclerk1 | EPI
ARV |
|
Administration (Reference Data) | admin administrator | All programs |
|
CCE | divo1 divo2 vsrmanager1 (supervises Cuamba) vsrmanager2 (one facility) | EPI |
|
Reporting | reporter1 reporter2 reporter3 | All programs |
|
Reporting Tests
The following Superset reports need to be tested:
Reporting Rate and Timeliness;
Stock Status;
Stockouts;
Consumption;
Orders;
Adjustments;
Administrative.
Each report is covered by one manual test case. Only one person should be executing a test case concerning the Superset reports at a time. They should inform others on the QA Slack channel that they have just started executing the test case, that they re-started the processor groups in NiFi, and that they finished executing the test case. Then, some other person can start executing another test case related to the Superset reports.
Exploratory Tests
Translations;
Edge case scenarios.
More details concerning this kind of testing can be found in the section on exploratory testing in the Testing Guide: https://openlmis.readthedocs.io/en/latest/conventions/testing.html#exploratory-testing.
Performance Tests
The performance testing scenarios are located here: Performance Metrics.
Enter the performance metrics here: https://docs.google.com/spreadsheets/d/1z1D4EUHsE-R_bUTt4HYcWiVDEy_UX50lZhdiyiC4bYg/edit#gid=0.
Deploying the release: Release Checklist 3.4.