Labsco
root-signals logo

Root Signals

β˜… 12

from root-signals

Equip AI agents with evaluation and self-improvement capabilities with Root Signals.

πŸ”₯πŸ”₯πŸ”₯βœ“ VerifiedAccount requiredAdvanced setup

Scorable logo

Measurement & Control for LLM Automations

Scorable MCP Server

A Model Context Protocol (MCP) server that exposes Scorable evaluators as tools for AI assistants & agents.

Overview

This project serves as a bridge between Scorable API and MCP client applications, allowing AI assistants and agents to evaluate responses against various quality criteria.

Features

  • Exposes Scorable evaluators as MCP tools
  • Implements SSE for network deployment
  • Compatible with various MCP clients such as Cursor

Tools

The server exposes the following tools:

  1. list_evaluators - Lists all available evaluators on your Scorable account
  2. run_evaluation - Runs a standard evaluation using a specified evaluator ID
  3. run_evaluation_by_name - Runs a standard evaluation using a specified evaluator name
  4. run_coding_policy_adherence - Runs a coding policy adherence evaluation using policy documents such as AI rules files
  5. list_judges - Lists all available judges on your Scorable account. A judge is a collection of evaluators forming LLM-as-a-judge.
  6. run_judge - Runs a judge using a specified judge ID

How to Contribute

Contributions are welcome as long as they are applicable to all users.

Minimal steps include:

  1. uv sync --extra dev
  2. pre-commit install
  3. Add your code and your tests to src/scorable_mcp/tests/
  4. docker compose up --build
  5. SCORABLE_API_KEY=<something> uv run pytest . - all should pass
  6. ruff format . && ruff check --fix