Flaky tests, and how to kill them
A test that fails sometimes is worse than no test at all — it teaches the whole team to ignore red. Here's how flakiness starts, and how to end it.
7 steps · 5 minWhat a flaky test costs
A flaky test passes and fails on the same code, at random. The first cost is time — reruns, investigations. The real cost is trust: once 'it's probably just flaky' becomes a normal sentence, real failures start getting waved through with it.
A suite the team doesn't believe is a suite that catches nothing.
A test fails on the build server, passes on rerun, and nobody can explain why. The team wants to ship. What's the right call?
Cause #1: timing
Most flakiness is a test racing the app. The test clicks 'Save' and immediately checks for the confirmation — but the server took 300ms longer than usual, so the check ran too early. Same code, different day, different speed, different result.
A test fails intermittently after clicking 'Save'. A teammate suggests adding a 5-second pause. What's the better fix?
Cause #2: shared state
Tests that share data poison each other. Test A renames the account; test B assumes the old name — B now fails only when A runs first. Symptoms: tests that pass alone but fail together, or fail only in parallel runs.
The cure is isolation: each test creates what it needs and cleans up after itself, owing nothing to the tests around it.
A test always passes when run alone but fails about half the time in the full suite. What does that pattern point to?
Treat flakes like bugs
A flaky test is a bug in the test suite, and it earns a bug's treatment: reproduce it (run it 50 times, run it in parallel), find the root cause, fix it properly. Teams that do this have suites people trust — which is the entire point of having one.
Next: where all these tests actually run — the CI pipeline.
Lesson complete.
You now treat a flaky test as a bug in the suite: reproduce it, fix the root cause — usually timing or shared state — and never train the team to ignore red.