toolsintegration-testingdockertestcontainersci

Testcontainers: real databases, Kafka and Redis in tests instead of mocks and a shared environment

Integration tests have two classic problems. The first — mocks lie: you replace a database or broker with a stub, the test is green, and production breaks, because the stub knows nothing about real transactions, indexes, deadlocks and message serialization. The second — the shared environment: one database for everyone, tests trampling each other’s data, flaking, and depending on what someone loaded yesterday.

Testcontainers solves both. It’s a library that, for the duration of a test, spins up real dependencies in Docker — a real PostgreSQL, Kafka, Redis, Elasticsearch, anything — lets the test connect to them, and tears everything down afterward. The test gets not a stub but exactly the software running in production.

What it looks like

The idea is simple: in the test code you declare a container with the image you need, the library starts it, maps the port to a random free one (so tests don’t fight over 5432), and gives you the host and port — you connect with the usual driver. After the test the container is killed.

It works from Java, Go, Python, Node.js, .NET, Rust and more — a single approach across all languages. Under the hood all you need is a running Docker (or a compatible runtime — Podman, Colima, Testcontainers Cloud).

Key patterns you can’t skip

1. Wait strategies — not sleep, but readiness

“Container started” and “ready to accept connections” are different moments. Postgres has launched the process but is still initializing the database. The classic beginner anti-pattern is sleep(5). The right way is a wait strategy: wait for a log line, an open port, a successful HTTP endpoint, or a healthcheck. Details in the startup & wait strategies docs.

2. Reusable / one container per class

Spinning up a fresh container for every test method is slow. Two recipes: keep one container per test class (it starts once, clean the data between tests), or enable reusable containers — the container survives the run and is reused locally for speed (usually disabled in CI). Between tests — truncate/rollback, not recreation.

3. Ryuk — automatic cleanup

Testcontainers starts a sidecar called Ryuk that kills leftover containers/networks/volumes if a test crashes or the process is killed. This saves CI agents from “leaked” containers clogging disk and memory.

4. Modules for specific technologies

There are ready modules: PostgreSQLContainer, KafkaContainer, RedisContainer / GenericContainer, MongoDB, MySQL, Elasticsearch, RabbitMQ, plus LocalStack (AWS emulation: S3, SQS, DynamoDB) and a WireMock module. For any image without a ready module — GenericContainer with explicit ports and a wait strategy.

5. Random ports are a feature

The container maps the internal port to a random external one. Never hardcode localhost:5432 — use getMappedPort() / getConnectionString(). Otherwise parallel tests will fight over the port.

6. Pin the image version

The image is part of the test. Don’t use latest: pin a tag (postgres:16.3) so production, tests and CI all speak about one version. Otherwise “it was green yesterday” turns into detective work.

CI: where teams usually trip

  • Docker-in-Docker / socket. The agent needs access to the Docker daemon (DinD or a mounted socket). On many SaaS CI providers this is a flag.
  • Image cache. The first run pulls images from the registry — slow. Cache layers/images between runs, otherwise every PR waits on downloads.
  • Resources. Heavy images (Elasticsearch, Kafka) eat RAM/CPU. Account for agent limits, or you get OOM and flakes.
  • Parallelism. Random ports let you run in parallel, but the total load on the agent grows — balance it.
  • Testcontainers Cloud. If there’s no local Docker or the agent is weak, you can offload containers to the provider’s cloud.

When Testcontainers is NOT needed

  • Unit tests. Pure logic with no external dependencies — a container only slows it down. Mocks are appropriate exactly here.
  • Very heavy startup. If an image takes a minute to boot and there are hundreds of tests — reconsider granularity (one container per class/suite, not per test).
  • Contract tests against third-party APIs. For external HTTP services a mock server (WireMock) or contract testing (Pact) is often better than booting someone else’s service.
  • E2E on a full environment. When you need the whole stack — that’s the job of staging/ephemeral environments, not a single test.

Rule: Testcontainers is for integrating with infrastructure (databases, brokers, caches, storage). It’s neither a replacement for unit tests below, nor for a full environment above.

Adoption checklist (10 points)

  1. Docker is available locally and on CI agents (DinD / socket / Testcontainers Cloud).
  2. Image versions are pinned by tag, not latest; they match production.
  3. Wait strategies (logs/port/HTTP/healthcheck) are used, not sleep.
  4. Connect via getMappedPort()/connection string, no hardcoded ports.
  5. One container per class/suite; between tests — truncate/rollback, not recreation.
  6. Reuse enabled locally for speed, disabled in CI for cleanliness.
  7. Ryuk isn’t disabled without reason — automatic cleanup works.
  8. Image cache configured on CI so it doesn’t pull every run.
  9. Agent resources sized for heavy images; parallelism balanced.
  10. Testcontainers is applied to infrastructure, not to unit logic and not to full E2E.

Bottom line

Testcontainers removes the main self-deception of integration tests — “we’re all green on mocks.” You test against a real Postgres with its transactions and a real Kafka with its message ordering, all without a shared environment or manual cleanup. The price is Docker on agents and a bit more resources. For most teams that’s a fair deal.

Sources