Bellwether

The open-source MCP testing tool. Catch drift before your users do.

What is MCP? MCP (Model Context Protocol) is how AI assistants like Claude connect to external tools—read files, query databases, call APIs. When those tool schemas change, AI workflows break silently.

Why Bellwether?

MCP servers expose tools with JSON schemas. When those schemas change—a parameter renamed, a type modified, a tool removed—AI agents break silently. Bellwether catches these changes before they reach production.

Product Focus

Bellwether is intentionally opinionated:

• Core workflow (default): init -> check -> baseline
• Advanced workflow (opt-in): explore, watch, discover, golden, contract, registry

If you only need CI-safe drift detection, you can stay entirely in the core workflow.

Two Modes

Most users only need check. It's deterministic, fast, and catches the changes that break AI agents.

CI/CD Workflow

Store your baseline in git. Run checks in CI. No account needed.

# 1. Initialize and save baseline (one-time setup)
bellwether init npx @mcp/your-server
bellwether check
bellwether baseline save
git add bellwether.yaml bellwether-baseline.json
git commit -m "Add Bellwether baseline"

# 2. Add to CI (.github/workflows/bellwether.yml)
name: MCP Drift Detection
on: [pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npx @dotsetlabs/bellwether check --fail-on-drift

What It Detects

Comparisons are protocol-version-aware — version-specific fields (annotations, titles, output schemas, etc.) are only compared when both baselines support the relevant MCP protocol version.

Command Tiers

Core Commands (Recommended)

Advanced Commands (Optional)

CI/CD Exit Codes

GitHub Action

- uses: dotsetlabs/bellwether@v2.1.3
  with:
    version: '2.1.3'
    server-command: 'npx @mcp/your-server'
    baseline-path: './bellwether-baseline.json'
    fail-on-severity: 'warning'

Environment Variables

Documentation

docs.bellwether.sh — Full reference for configuration and commands.

• Quick Start
• Core vs Advanced
• CLI Reference
• CI/CD Integration
• Golden Paths
• Compatibility Policy

Project Governance

• Changelog
• Releases

Development

git clone https://github.com/dotsetlabs/bellwether
cd bellwether
npm install
npm run build
npm test

Quick Start

npm install -g @dotsetlabs/bellwether
bellwether init npx @mcp/your-server
bellwether check

That's it. No API keys. No LLM costs. Runs in seconds.

Configuration

All settings live in bellwether.yaml. Create one with presets:

bellwether init npx @mcp/your-server           # Default (free, fast)
bellwether init --preset ci npx @mcp/server    # Optimized for CI/CD
bellwether init --preset local npx @mcp/server # Local Ollama (free)

For remote MCP servers that require auth headers, configure:

server:
  transport: sse
  url: "https://api.example.com/mcp"
  headers:
    Authorization: "Bearer ${MCP_SERVER_TOKEN}"

Or use one-off CLI overrides:

bellwether check -H "Authorization: Bearer $MCP_SERVER_TOKEN"

License

MIT License - see LICENSE for details.

Built by Dotset Labs