get_flaky_testsLABSCO SUMMARY
One trace tells you what failed; this tells you what keeps failing.
A single failed run is ambiguous. Accumulated history turns it into a fact — this test fails one run in six, only on Firefox, and started doing it after a particular commit.
What it is
A Playwright custom reporter paired with an MCP server. The reporter records every run into a local SQLite database; the server lets an agent query that history for flakiness rates, failure patterns by browser and OS, and the commits they line up with.
What you get
- Tests ranked by flakiness rate — failed plus flaky over total runs
- Full run history for one test: status, duration, error, retry, browser, OS
- Failure rates broken down by browser and OS combination
- Tests ranked by average duration
- Failures clustered by exact error prefix to surface shared root causes
- A flakiness trend over time
- Flakiness correlated against git commits
- Errors clustered semantically when the messages differ
Requirements
A Playwright project with the custom reporter added, and the SQLite database path passed as db_path on every call. Installed from npm as flakiness-knowledge-graph-mcp.
Setup effort
One command — npm install -g flakiness-knowledge-graph-mcp
USE CASES
01
Deciding which test to fix first
Rank tests by flakiness rate over the runs recorded so far.
02
Working out whether a failure is new
Read that test's full history — status, duration, error, retry, browser and OS.
03
Finding a browser-specific problem
Break failure rates down by browser and OS combination.
04
Spotting one root cause behind many failures
Cluster failures by error prefix, or semantically when the messages are worded differently.
05
Finding the commit that started it
Correlate flakiness against git history instead of guessing at a release.
06
Watching the suite get slower
Rank tests by average duration and read the flakiness trend alongside it.
8 tools
Listed in project docs. Your client may show a different list.
get_test_historyget_failure_patternsget_slow_testsget_error_groupsget_flakiness_trendcluster_semantic_error_treescorrelate_git_commit_flakinessBEFORE YOU INSTALL
What to know first
Tools you get
8 tools named in the project's documentation.
Requires
flakiness-knowledge-graph-mcp on PATH
INSTALLATION
Connect your client
Server setup
TERMINAL
npm install -g flakiness-knowledge-graph-mcpMCP Clients
Cursor
CURSOR · JSON
{
"mcpServers": {
"flakiness-knowledge-graph": {
"command": "flakiness-knowledge-graph-mcp"
}
}
}VS Code
VS CODE · JSON
{
"servers": {
"flakiness-knowledge-graph": {
"command": "flakiness-knowledge-graph-mcp"
}
}
}Claude Code
CLAUDE CODE · TERMINAL
claude mcp add flakiness-knowledge-graph flakiness-knowledge-graph-mcp
