The 3.9 Regression and Release Candidate Test Plan
The Release Candidate Test Plan
When all 3.9 tickets have been completed, we can begin testing, and we will follow the test plan detailed below.
Before we start the Release Candidate testing:
- If there are any blocker or critical bugs when we start Sprint 146, they must be fixed and tested before the RC testing can begin;
- If there are any open tickets related to the features in the 3.9 release, they must be completed and marked as Done before any RC testing begins;
- Regular regression testing has been completed in previous sprints before we start the release candidate testing.
Starting the Release Candidate testing:
- If there are no open blocker or critical bugs, then move to the release candidate's deployment and testing:
- RC1 (Three full days of testing, and 3 days for bug fixes and bug fix testing) for each release candidate as needed;
- The testing will be done in the following environments:
- Team Mind the Parrot's UAT environment: https://uat.openlmis.org;
- Performance testing: https://perftest.openlmis.org.
- How to enter bugs: Testing Process & Test Plans#EnterDefectsduringsprinttesting;
- Prioritization of bugs: http://docs.openlmis.org/en/latest/contribute/contributionGuide.html#reporting-bugs.
Test Phase | Components/Features | Assigned in Test Cycle | Dates | QA lead responsibilities |
|---|---|---|---|---|
| RC 1 |
| 3 full days (as needed) | Team Mind the Parrot:
| |
| Bug triage | Bug triage for RC1 | Bug fixes test cycle for a given Release Candidate, e.g. 3.9 RC1 Bug fixes | 3 full days (as needed) |
|
Suggested schedule, assuming we start testing on March 12:
Week One:
| Thu Mar 12 | Fri Mar 13 | Mon Mar 16 | Tue Mar 17 | Wed Mar 18 |
|---|---|---|---|---|
|
|
|
|
|
Week Two:
| Thu Mar 19 | Fri Mar 20 | Mon Mar 23 | Tue Mar 24 | Wed Mar 25 |
|---|---|---|---|---|
|
|
|
|
|
Week Three:
| Thu Mar 26 | Fri Mar 27 | Mon Mar 30 | Tue Mar 31 | Wed Apr 01 |
|---|---|---|---|---|
Week Four:
| Thu Apr 02 | Fri Apr 03 | Mon Apr 06 | Tue Apr 07 | Wed Apr 08 |
|---|---|---|---|---|
Roles & Responsibilities
QA Team Leads
Joanna Bebak (Deactivated) will be the QA lead for the Mind the Parrot team.
Owner | Responsibilities | Questions/Tasks before release begins |
|---|---|---|
| ||
Team leads: |
|
|
| Team responsibilities |
|
|
The Bug Triage team
Members | Responsibilities | Questions |
|---|---|---|
|
|
Communication on the Test Plan and the daily testing status:
- What needs to be communicated daily?
- The test cycle execution status (including the test cycle's name and % of completion);
- # of test cases that were executed, passed and failed;
- Joanna Bebak (Deactivated) will post any test cases or items that need attention and review by the end of her day on the QA Slack channel;
- Joanna Bebak (Deactivated) will communicate with the Malawi team (via the Malawi Slack channel) and notify them about the release candidate testing:
- The communication of the scheduled start date before the release candidate testing begins;
- The communication of when we start the release candidate testing (Malawi has one week to test);
- The Malawi team is included in the daily bug triage meetings.
- The best time of day (for each team to communicate the morning status and the end of day status & share blockers):
- The beginning of the day: Post what we are doing today;
- The end of day: Post the status of what we have done, and anything pending.
Environments and the demo data for testing
Environments: uat.openlmis.org.
Refer to the demo data readme for more details about user permissions: https://github.com/OpenLMIS/openlmis-referencedata/blob/master/src/main/resources/db/demo-data/README.md.
ONLY test with the users "admin" or "administrator" when executing test cases related to administrative activities.
Test Data
Component | Username | Program | Concerns |
|---|---|---|---|
Requisition | srmanager1, smanager1, psupervisor, wclerk1 srmanager2, smanager2, psupervisor, wclerk1 srmanager4 (for the second approval), smanager4, dsrmanager, psupervisor chaz (for the second approval) administrator (testing requisition template updates/changes or program settings changes) | Family Planning Essential Meds ARV Essential Meds and Family Planning ARV Family Planning Essential Meds EPI ARV |
|
| Stock Management | srmanager2 divo1, rivo | Family Planning, Essential Meds EPI | |
| Fulfillment | vsrmanager1 vsrmanager2 divo1 divo2 rivo vwclerk1 wclerk1 | EPI ARV | |
| Administration (Reference Data) | admin administrator | All programs | |
| CCE | divo1 divo2 vsrmanager1 (supervises Cuamba) vsrmanager2 (one facility) | EPI | |
| Reporting | administrator | All programs |
Reporting Tests
The following Superset reports need to be tested:
- Reporting Rate and Timeliness;
- Stock Status;
- Stockouts;
- Consumption;
- Orders;
- Adjustments;
- Administrative.
Each report is covered by one manual test case. Only one person should be executing a test case concerning the Superset reports at a time. They should inform others on the QA Slack channel that they have just started executing the test case, that they re-started the processor groups in NiFi, and that they finished executing the test case. Then, some other person can start executing another test case related to the Superset reports.
Exploratory Tests
- Translations;
- Edge case scenarios.
More details concerning this kind of testing can be found in the section on exploratory testing in the Testing Guide: https://openlmis.readthedocs.io/en/latest/conventions/testing.html#exploratory-testing.
Performance Tests
The performance testing scenarios are located here: Performance Metrics.
Enter the performance metrics here: https://docs.google.com/spreadsheets/d/1z1D4EUHsE-R_bUTt4HYcWiVDEy_UX50lZhdiyiC4bYg/edit#gid=0.
Deploying the release: Release Checklist 3.4.