The aim of this document is to gather guidelines to make releasing process faster. This document is supposed to point out what should be tested by manual, E2E and contract tests in order to offer a higher level of general testing automation. It should save our time spent on regression testing for release candidates and make the process faster.
...
- Update test strategy to include a requirement to periodically review test cases (similar to how regression tests are scheduled for every other sprint) - discussion item on QA call
- Spike / research on how to improve searching for existing test cases / test steps to more easily identify whether a feature/case is already covered by manual tests and to prevent duplicating test cases / steps (ticket)
- TODO: Improve test strategy to more clearly explains/give examples on what should be checked/tested in manual steps and what shouldn't (ticket to move thoughts from this page to official docs and prepare examples)
- Update developer documentation to include that quality is not only a concern of testers and that the committed changes are already tested by the developer before the ticket makes it to the QA column (ticket to update docs)
...
Currently, test cases are run in a context of a change that is happening in the given ticket and they ignore the fact that other parts of the system could be affected. Say a work on a Stock Management screen required a change to Orderables schema or API. The developer makes the change and the Stock Management screen works as expected, what is also verified with a test case. However, a change to Orderables schema or API affects many other places in the system that are not covered by that test case. We expect to catch the obvious and biggest issues with automated tests. For edge cases, we should have a better process to define which parts of the system are affected by a ticket and then an easy way to find all the test cases related to them.
Next steps:
- TODO: Spike/research on how developers should mark what parts of the system were affected by the change in a ticket - ticket for research and updating docPoC: Contract tests that verify from the UI perspective
- Review new test cases as they are created - who should do this? Other testers? Developers? BA? Product Owner? - discussion item for QA call
...
The value of manual tests for happy paths is not worth the time we need to invest in executing and creating those tests. Executing a manual test case with 15 steps usually lasts about 1h, while creating a new manual test case takes about 2 hours. We can save a lot of time replacing them with functional tests. In the case of edge cases, the value is significant. They are quite rare and the work needed to rewrite them to automatic tests is incommensurate.
In contrast, there are such benefits of automated tests as reducing lead time and increasing the number of deployments as releasing time would last significantly less. Leading companies/products are able to release several times a day.
Next steps:
- TODO: Define strategy for existing and new manual test cases that verify very specific UI elements/behaviour (eg. input placement, resizing, sticky columns, loading icon, drag&drop) - create a ticket to update test strategy and update/dead no longer relevant TCs
...
On the other hand, this assumption should not relate to all the edge cases. Especially at the beginning of automating test cases, we should try to automate both happy paths and edge cases. This document should be updated after having written the fist test for any edge case to provide decision about our edge-cases-approach.
- TODO: Document what is covered by what type of tests - new ticket or update the below ticket
- Merge testing documents into one (
)Jira Legacy server System JIRA serverId 448ba138-230b-3f91-a83e-16e7db1deed1 key OLMIS-5388
...