2018-07-03 QA Weekly Meeting notes

Test Plan for 3.4 release:

• Regression testing
• Performance testing
• Release candidate testing
• Malawi UAT

Regression Testing: Will we have time/resources for regression testing before the release candidate testing? 

• Concern: If we don't do regression testing then we have a risk of introducing breaking changes during the last few sprints. 
• Proposal: Regression test specific components
  • Concern: Regression testing being done during active development is not preferred
• Concern: Resources to test during regression
  • Proposal: Leave some capacity for bug fixes for regression testing during a sprint
  • Proposal: organization of team to spend more time focusing on regression testing during the sprint to relieve the QA (and allow them to focus on testing new features)
• Concern: Sam is out of office mid-July (during regression testing), Joanna is on vacation mid-July (9-20)
  • Proposal: Regression testing only during the last few days of the sprint, if new feature work is done, it may be more impactful if we focus on performance testing
  • The same tests run during regression cycles are part of the release candidate process
  • Proposal: The last sprint before we start release candidate process should include some capacity for regression testing of specific new features to reduce the risk of regression. If regression bugs are found, then they must be resolved within the sprint.
• Decision: We are not focusing on regression testing for the sprints before the release candidate, but if there is capacity we decide on using proposals listed above.

Performance testing: Team Parrot to provide guidance on where we can improve performance before 3.4 release: Convert to order, Approve, and "Initiating" Proceed on the requisition screen.

• Some performance improvements have been implemented as the team has worked on tickets
• Loading periods has performance improvements (this impacts requisition submit & initiate)
• Some performance improvements around creating an order (this impacts convert to order)
• For Sprint 103: Team Parrot will complete performance testing for Convert to Order & Initiating Proceed 

Release Candidate Testing:

• Release testing should include all teams testing during the same sprint period
• Concern: There have been multiple teams working on one component (requisitions) there isn't a clear way to break out the release of features by team
• Concern: Managing multiple teams, teams testing in parallel
  • End of day reports of testing status
  • More servers for testing
  • Automate testing status reports (when possible)
• Concern: Who fixes bugs found during regression & release testing
  • During Bug triage, identify and assign bugs to teams
  • Representatives from each team should be included in the bug triage to estimate and discuss capacity to fix
• Concern: Teams don't have work to do during testing (blocked by waiting for bug fixes)
  • Plan work for each team in case they are done early
    • Updating documentation, prepping release documentation

Patch Release process

• How does the patch release process impact

3.3.1 release will be before 3.4

• What do we need to test for the patch release? Can this be done within a sprint, with limited resources (to avoid disrupting ongoing development)?

3.3.1 release includes the fix for Malawi bug:

• Testing for this Malawi bug fix may be difficult because it is not always reproduced
• Testing would be verification that the fix has not broken anything else
• Happy path requisition test cases
• Patch release testing would focus on testing the component, but there is still a risk that other components are impacted by the bug fix
  • Default testing by component unless we know that the bug fix impacts other components
  • Testing would be determined by team (devs and QA) to ensure complete testing for the patch release
• Malawi would still conduct their own UAT for the patch release
• Are we planning to have release candidates for the patch release?
  • Since we are only focusing on a few bug fixes for the patch release, then it doesn't make sense to have a release candidate process for a patch release
  • If we fix bugs and deploy a patch release, and new bugs are found, what is the process?
    • Another patch release 3.3.2
    • Or Malawi doesn't take the patch release and waits for the 3.4 release
• For 3.4 release, dev teams should not work on new features, they should still focus on release candidate testing and bug fixes
• Feature flags will need to be in use before we can get to a point where dev teams can work on new features during the testing process
• For 3.4 release testing, devs should not work on any new features that would need to be pushed to the reporting repositories, or it could cause bugs that will need additional work