Test data in automated tests: a source of flakiness and coupling

When automated tests flake, people blame locators and timing first. But one of the most common and most invisible sources of instability is test data. A test that depends on a record in a shared database will eventually fail: someone changed it, deleted it, or two parallel tests fought over the same email.

Let’s look at how to prepare data so tests are isolated, deterministic, and survive parallel runs.

The main pain — shared mutable data

The classic scenario: there’s a “user [email protected]” on the environment, and a dozen tests depend on it. One test edits their profile, another deletes it, a third expects it in its original state. As long as tests run sequentially and in the same order — you get lucky. Turn on parallel runs or change the order — a cascade of failures you can’t reproduce locally.

Rule: a test must not depend on data it didn’t create itself. Any shared mutable record is a future flake.

Three ways to get data (and when to use which)

Fixtures / seed scripts: a pre-prepared set. Good for reference data (countries, plans) — immutable data. Bad for mutable entities that a test modifies.
Factories / builders: code creates the needed entity in the needed state right in the test (Object Mother, Test Data Builder). Flexible, readable: aUser().withSubscription(EXPIRED).build(). The best default for most tests.
Setup via API: create preconditions not through the UI but via an API/service-layer call — fast and reliable. Use the UI only for the scenario under test, not for data prep.

Anti-pattern — preparing data through the same UI you’re testing: slow, brittle, and mixes setup with verification.

Isolation: every test on its own

Each test creates its own and doesn’t rely on data “from last time”.
A fresh fixture (fresh state per test) is preferable to a shared fixture that tests silently mutate.
No chains of “test 1 created — test 5 uses”: tests must pass in any order and individually.

Determinism: remove randomness and time

Time: freeze “now” (clock/freeze), or tests for “over 18” or “expired yesterday” will drift at midnight and in a leap year.
Randomness: Faker (and in Python) is a great generator, but with a fixed seed. A “random name” will sometimes produce an empty string, an apostrophe, or 60 characters — and the test flickers. Seeded data is reproducible.
Order: don’t rely on the order of a DB query without ORDER BY — it’s not guaranteed.

Uniqueness: survive parallel runs

Under parallel tests the main enemy is collisions on unique fields: two tests create [email protected] → conflict. The fix — build uniqueness into the data: user+{uuid}@test.com, names with a timestamp/worker-id suffix. Then tests don’t fight over the same values.

Cleanup: don’t leave garbage

Teardown: the test deletes what it created at the end. Downside — on a crash before teardown, garbage remains.
Transactional rollback: wrap the test in a transaction and roll back. Fast and clean, but doesn’t work for multi-service scenarios and committed data.
Ephemeral DB: spin up a clean database per run (e.g. via Testcontainers — see our Testcontainers post). Maximum isolation, at the cost of resources.
Unique data (see above) partly removes the cleanup question: even without cleanup, you don’t disturb the next runs.

Prod-like data vs synthetic

Synthetic (factories/Faker) is the main source: controllable, safe, deterministic.
A prod dump “as is” into tests is dangerous: it’s PII/personal data (GDPR), and it also ties tests to the random state of production.
If you need realistic volume/distributions — anonymize/mask the dump (names, emails, cards), don’t drag in raw prod.

Anti-patterns (red flags)

Hardcoded ids/“magic” records from the environment (userId = 12345) — breaks when the environment is recreated.
Chains of dependent tests and a shared mutable user.
sleep(3) waiting “until the data appears/indexes” — wait on actual readiness (poll a condition), not on the clock.
Preparing data via the UI instead of the API.
“Random” data without a seed → flickering tests that blame the code.
Tests that pass only in a specific order.

Test data checklist (12 points)

The test creates its own data and doesn’t depend on “the previous run”.
Fresh fixture by default; a shared mutable set only for immutable reference data.
Preconditions are prepared via API/factories, not the UI under test.
Factories/builders instead of copy-pasting objects (Object Mother / Test Data Builder).
Time is frozen (freeze clock) where logic depends on dates.
Randomness via Faker with a fixed seed; nothing non-reproducible.
Unique fields contain uuid/worker-id to avoid collisions in parallel.
Cleanup is defined: teardown / transaction / ephemeral DB.
Tests pass in any order and individually (randomize order in CI).
No hardcoded ids from the environment.
Waiting for data readiness is by condition, not sleep.
Prod data in tests only anonymized; no raw PII.

Bottom line

The stability of automated tests rests not only on good locators and auto-wait, but on discipline with data. Isolation (own data per test), determinism (fixed time and seed), and uniqueness (uuid in fields) remove a whole class of “flickering” failures otherwise written off as “eh, it’s just flaky”. Prepare data in code, clean up after yourself, don’t drag in raw prod.

Test data in automated tests: a source of flakiness and coupling — and how to prepare it