LabscoMCP early access ↗
jurek-fMCP SERVER

Prompt Lab

Runs prompt-optimization and regression loops inside Claude Code: the agent does the scoring and rewriting, the server holds the workspace and keeps a web UI in sync as each iteration lands.

by jurek-f ↗LLM Evaluation & ObservabilityCommunity

LABSCO SUMMARY

The loop stops when every case passes, not when the average looks good.

The split is the interesting part — no model is called by the server, so your own agent and your own key do the scoring, and the server is just shared state plus a live view. The regression loop's rule that one failing case fails the run is what makes it usable as a guardrail rather than a demo.

What it is

An MCP server you run yourself that owns state for a prompt-tuning session — the system prompt, the test cases, the scored results, the pending suggestions — while the agent in your editor does all the LLM work. A companion web UI shows each iteration's score, the agent's reasoning and the revised system prompt as they land.

What you get

  • Session setup — start_web_app returns the UI URL, register_api_key registers a provider key for test runs, list_models and set_test_model pick the model, delete_session clears everything
  • Workspace state — get_workspace_state, set_system_prompt, add_test_cases, post_test_result, post_prompt_suggestion, apply_suggestion
  • Optimization — start_optimization_session for a single pass that stops for your review, loop_optimization to iterate until every score clears the threshold
  • Regression — run_regression_testsuite for one scored pass with no prompt changes, and loop_regression, which requires every individual case to pass rather than accepting a good average
  • Templates — save_template and save_system_prompt_template put reusable suites and prompts into the UI dropdowns
  • Archive — pull_ui_history fetches session summaries and past regression runs

Requirements

Claude Code, Node.js 18 or later and a local copy of the server: npm install, then npm run dev serves the MCP endpoint at http://localhost:3000/mcp for your .mcp.json. The author's hosted endpoint no longer answers. Provider keys are never stored in the MCP config: export ANTHROPIC_API_KEY, GEMINI_API_KEY or OPENAI_API_KEY into the agent's environment and it registers them per session, with the provider detected from the key prefix. UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN are optional; without them workspaces live in memory and are lost when the server stops. Set PROMPT_LAB_UI_URL to your own Prompt Lab UI (the mcp branch of github.com/jurek-f/prompt-lab, configured to reach this server) so the link start_web_app returns opens your workspace; unset, it links to the author's hosted UI at prompt-lab-mcp.vercel.app. MIT licensed.

Setup effort

Build from source — git clone https://github.com/jurek-f/prompt-lab-mcp.git

See installation options