|
We want to improve the process of writing contract tests to have more valuable contract tests. We also don't want to compromise doing demos of the system. The way contract tests load data must be improved in order to make them easier to extend and maintain - if we decide on not using demo data for contract tests, we will still have to think up with an alternative in order to have the contract tests more valuable.
Currently contract tests use their own set of data on which contract tests are run. This includes users, role assignments, facilities, requisition groups, supervisory nodes etc. This data is currently stored as hard to main sql data files (Link to GitHub directory). Editing these files is hard, coming up with extensions to them is hard, reading them is hard. Because of this, creating contract tests is a painful chore and makes writing a contract a task on itself - unlike writing unit/integration tests is pretty much perceived as part of the original task. In order to make our contract tests easier to extend going forward, with the hope that this will increase our coverage, we must take some steps to sanitize this data and make it more manageable.
A fairly easy to achieve way of doing this is starting to use our regular demo data as the base for contract testing. There are several pros to this, however care must be taken in order to not create undesired effects, such as causing issues to people giving demos of OpenLMIS.
Test data is reloaded before each scenario.
|
|
Switching contract tests to use demo-data should be easy - some changes need to be made to compose files in order to load the demo data when starting and the code that is currently responsible for loading the contract test data. The bigger part of this task will be going through our existing contract tests and switching them to use the existing data (primarily ID switching). There shouldn't be an issue with data missing since demo data is more robust and provides the more complex hierarchies and users - the tests will have to be adjusted to use the new data.
Currently data in contract tests is reloaded before each scenario runs - this is something we probably would want to continue having in order to run tests in isolation. With how we currently load demo data, this might not be obvious. However here is a possible solution that is easy to achieve - after the distro is ready, but before running the first test, take a snapshot (dump) of the database. Then revert to that database before each test scenario. That way demo data lives in the services, but contract tests keep reloading it.
To summarize - we will need to create two tickets: one for switching to demo data and handling data reloads, the second one to rewrite actual contract tests to use new ids.