How to Fix Flaky Tests: 10 Proven Strategies for Reliable Test Automation
If you've ever seen a test fail in your CI pipeline, rerun it, and watch it pass without changing a single line of code, you've encountered a flaky test.
Flaky tests are one of the biggest productivity killers in modern software development. They waste engineering time, slow down releases, and make teams lose trust in their automated testing pipelines.
Understanding how to fix flaky tests is essential for building reliable software. Instead of masking failures with retries, successful engineering teams focus on removing the root causes that make tests non-deterministic.
In this guide, we'll explore why flaky tests occur and the practical techniques you can use to eliminate them.
What Are Flaky Tests?
A flaky test is a test that produces inconsistent results without any changes to the application code.
For example:
Passes locally
Fails in CI
Passes again after rerunning
Fails randomly the next day
The application hasn't changed, but the outcome has.
This unpredictability makes flaky tests difficult to trust.
Why Flaky Tests Matter
Many teams simply rerun failed tests and move on.
While this may temporarily unblock deployments, it creates larger problems over time.
Flaky tests can lead to:
Slower release cycles
Reduced confidence in CI pipelines
Hidden production bugs
Increased debugging time
Higher infrastructure costs
Delayed developer productivity
Ignoring flaky tests often causes engineering teams to stop trusting their entire test suite.
Common Causes of Flaky Tests
Most flaky tests are caused by non-deterministic behaviour.
Some common reasons include:
Race conditions
Network latency
Asynchronous operations
Shared databases
External APIs
Random test data
Time-dependent logic
Environment differences
Resource contention
Identifying the actual cause is the first step toward fixing the problem permanently.
1. Replace Fixed Waits with Explicit Waits
Hardcoded delays are one of the most common reasons for flaky tests.
Instead of waiting a fixed number of seconds, wait until the application reaches the expected state.
Instead of:
await sleep(5000);
Use:
await page.waitForSelector("#dashboard");
Explicit waits improve both reliability and execution speed.
2. Make Every Test Independent
Tests should never depend on another test.
Avoid sharing:
Sessions
Cache
Database records
Temporary files
Global variables
Each test should create its own data and clean up afterward.
Independent tests can run in any order without affecting each other.
3. Remove External Dependencies
Calling real APIs during testing introduces unnecessary instability.
External systems may experience:
Downtime
Slow responses
Authentication failures
Rate limits
Network interruptions
Instead, replace unstable dependencies with deterministic responses during testing.
4. Use Predictable Test Data
Random data often causes inconsistent behaviour.
Avoid generating:
Random usernames
Random timestamps
Random IDs
Random ordering
Instead:
Use fixed datasets
Seed random generators
Freeze timestamps
Predictable inputs create predictable outputs.
5. Properly Handle Asynchronous Operations
Applications frequently perform work in the background.
Examples include:
Database writes
Queue processing
API requests
Event listeners
Assertions should only execute after asynchronous operations complete.
Waiting for actual completion events is far more reliable than using arbitrary delays.
6. Keep Test Environments Consistent
Many flaky tests only fail inside CI.
This usually indicates differences between environments.
Keep environments consistent by:
Pinning dependency versions
Using containers
Matching environment variables
Standardizing runtime versions
The more similar environments are, the fewer surprises you'll encounter.
7. Avoid Shared Resources
Parallel execution is now common in CI pipelines.
If multiple tests use the same:
Database
Port
File
Cache
they may interfere with each other.
Each test should operate in complete isolation.
8. Remove Time-Based Dependencies
Tests that rely on the current time often become unstable.
Examples include:
Expiring tokens
Scheduled jobs
Time zone calculations
Date comparisons
Mock or freeze time whenever possible to make test execution deterministic.
9. Investigate Every Intermittent Failure
A flaky test isn't "just CI."
Every intermittent failure indicates either:
A bug in the application
A bug in the test
A problem with the environment
Treat flaky failures like production issues and investigate their root cause instead of ignoring them.
10. Continuously Maintain Your Test Suite
Applications evolve continuously.
Tests should evolve too.
Regularly:
Remove obsolete tests
Refactor duplicated code
Update outdated fixtures
Review unstable scenarios
A healthy test suite requires ongoing maintenance.
Common Mistakes When Fixing Flaky Tests
Many teams accidentally make the problem worse.
Some common mistakes include:
Adding Retries
Retries hide flaky behaviour instead of fixing it.
Increasing Wait Times
Longer delays slow down the pipeline without guaranteeing stability.
Mocking Everything
Overusing mocks reduces confidence in integration testing.
Only external dependencies should typically be mocked.
Ignoring Random Failures
Every intermittent failure deserves investigation.
Small issues often become major reliability problems later.
Best Practices Checklist
Before merging a new test, verify that it:
Runs independently
Doesn't depend on execution order
Uses deterministic data
Waits for application state
Doesn't rely on arbitrary delays
Cleans up after execution
Handles asynchronous operations correctly
Avoids unstable external dependencies
Following these practices significantly reduces flaky behaviour over time.
Final Thoughts
Learning how to fix flaky tests is about making your tests deterministic, repeatable, and independent.
Rather than relying on retries or increasing wait times, focus on eliminating shared state, reducing environmental differences, handling asynchronous operations correctly, and isolating unstable dependencies.
A reliable test suite gives developers confidence to ship faster, catch regressions earlier, and spend less time debugging false failures. Over time, these improvements lead to more stable CI/CD pipelines and a smoother development workflow.
About the author
- I read RFCs for fun, debug APIs before coffee, and write about testing, APIs, and DevTools. If it breaks in production, I want to know why
Comments
Join the discussionNo comments yet. Be the first to comment.