Summary
One trace tells you what failed; this tells you what keeps failing.
A single failed run is ambiguous. Accumulated history turns it into a fact — this test fails one run in six, only on Firefox, and started doing it after a particular commit.
What it is
A Playwright custom reporter paired with an MCP server. The reporter records every run into a local SQLite database; the server lets an agent query that history for flakiness rates, failure patterns by browser and OS, and the commits they line up with.
What you get
- Tests ranked by flakiness rate — failed plus flaky over total runs
- Full run history for one test: status, duration, error, retry, browser, OS
- Failure rates broken down by browser and OS combination
- Tests ranked by average duration
- Failures clustered by exact error prefix to surface shared root causes
- A flakiness trend over time
- Flakiness correlated against git commits
- Errors clustered semantically when the messages differ
Requirements
A Playwright project with the custom reporter added, and the SQLite database path passed as db_path on every call. Installed from npm as flakiness-knowledge-graph-mcp.
Setup effort
One command — npm install -g flakiness-knowledge-graph-mcp