Many testing problems are really test data problems. A test fails because a customer account is missing, another team changed the data, or the test environment has not been refreshed for months. On top of that comes the question of whether real customer data may be in the test environment at all. Test data management makes sure the right data is available at the right time, in a way that keeps tests reliable and privacy-compliant. This article explains the types of test data, what to watch out for regarding data protection, and how to build a test data strategy in five steps.
Test data management covers everything to do with the data used for software testing: determining what is needed, generating or obtaining test data, provisioning it in the test environments, keeping it current and finally deleting it. Test data is not just input values in a form, but also the data a test builds on: customer accounts, contracts, products, orders or master data from connected systems.
The general approach to test data belongs in the test strategy. The concrete data a single test needs is specified in the test case. Test data management connects both levels and makes sure the decisions are actually put into practice in day-to-day project work.
In many projects the topic only gets attention once it slows things down. Typical signs:
There is no single right source for test data. In practice, several types are combined depending on the test level and the risk.
Synthetic data is generated artificially, for example by script, with a data generator or through the application's own interfaces. It contains no real personal data, can be created specifically for boundary values and edge cases, and can be regenerated as often as needed. Its limit: it is only as realistic as it was modelled. Unusual constellations from legacy data that has grown over years are often missing.
Here, an extract of production data is altered so that no conclusions about real people are possible: names, addresses, account numbers or e-mail addresses are replaced without destroying formats and business relationships. The advantage is realistic data variety. The effort lies in masking that is truly effective and keeps related data consistent across several systems.
Instead of a full copy, only a coherent business extract is taken over, such as selected customers with all their contracts and bookings. This saves storage and loading time and is usually combined with masking.
For individual business tests, for example during acceptance, testers often create data directly through the user interface. This is flexible but hard to reproduce. Manually created data should therefore be documented or later replaced by scripts.
If production data contains personal data, the GDPR also applies in the test environment. An unchanged copy of real customer data in a test environment is therefore usually problematic, especially as more people tend to have access there and protective measures are often weaker than in production.
The difference between anonymisation and pseudonymisation matters. Pseudonymised data can be linked back to a person with additional information and therefore remains personal data. Only effectively anonymised data falls outside the GDPR. In practice this means:
These notes are not legal advice, but they show where in the project the decisions need to be made.
Start from the test cases and test scenarios: which data constellations are needed? For an online shop, for example, new customers, existing customers with an open invoice, locked accounts and items out of stock. This list is the basis for everything else.
Synthetic data is usually sufficient for component and integration tests. System and acceptance tests often need a more realistic data basis, in which case anonymised production data is used. For load tests, a production-like data volume matters most.
Test data generated by script or through interfaces is ready in minutes instead of days. Keep these scripts under version control like code, so they evolve along with the application.
Decide which data belongs to whom and how a defined starting state is restored, for example by resetting the database before a test run or by giving each team its own records. This keeps results repeatable.
Test data ages: data models change, new products are added. Name a person responsible, plan regular refreshes and delete data that is no longer needed. In a project, this is a typical task of test management.
Stable test data is a prerequisite for test automation. A proven approach is for each automated test to create its own data, for example via an API, and remove it after the run. Tests then become independent of each other and can run in parallel. Where this is not possible, a fixed baseline that is reset before every run helps. This pays off especially for regression tests that run before every release: failures then point to real defects rather than data problems.
In Scrum teams, the question of test data should already be raised during refinement: which data does this user story need, and does it already exist? Teams that only clarify this during the sprint lose time there. It helps to include the provision of the required test data in the Definition of Done. Read more about testing in the sprint in the article Agile Testing in Scrum.
Good test data management goes almost unnoticed; poor test data management is noticed all the more. If you derive data requirements from the tests, choose the right source for each test level, automate provisioning and consider data protection from the start, you get repeatable tests, shorter waiting times and fewer discussions about whether a failure lies in the code or in the data.
Test data management covers planning, creating, provisioning, maintaining and deleting the data needed for software testing. The goal is that every test level has suitable, up-to-date and privacy-compliant test data at the right time, without tests interfering with each other.
If production data contains personal data, the GDPR also applies in testing. An unchanged copy in a test environment is therefore usually problematic. Common approaches are synthetic data or production data that has been effectively anonymised or masked beforehand. Pseudonymised data remains personal data. The specific approach should be agreed with the data protection officer.
Synthetic test data is generated artificially, for example by script, with a data generator or through the application's own interfaces. It contains no real personal data, can be created specifically for boundary values and edge cases, and can be reproduced at will. Its drawback: it only reflects the variety of real data as well as it was modelled.
30-minute initial consultation — free and without obligation. Together we look at your test environments and test data and show where your tests can become more reliable.
Book a Free Initial ConsultationMore about our service: Software quality assurance and test management in Hamburg →