The split is the interesting part — no model is called by the server, so your own agent and your own key do the scoring, and the server is just durable shared state plus a live view. The regression loop's rule that one failing case fails the run is what makes it usable as a guardrail rather than a demo.
A hosted MCP server that owns state for a prompt-tuning session — the system prompt, the test cases, the scored results, the pending suggestions — while the agent in your editor does all the LLM work. A companion web UI reads the same workspace, so you watch iterations arrive live.
- Session setup — `start_web_app` returns the UI URL, `register_api_key` registers a provider key for test runs, `list_models` and `set_test_model` pick the model, `delete_session` clears everything
- Workspace state — `get_workspace_state`, `set_system_prompt`, `add_test_cases`, `post_test_result`, `post_prompt_suggestion`, `apply_suggestion`
- Optimization — `start_optimization_session` for a single pass that stops for your review, `loop_optimization` to iterate until every score clears the threshold
- Regression — `run_regression_testsuite` for one scored pass with no prompt changes, and `loop_regression`, which requires every individual case to pass rather than accepting a good average
- Templates — `save_template` and `save_system_prompt_template` put reusable suites and prompts into the UI dropdowns
- Archive — `pull_ui_history` fetches session summaries and past regression runs
Claude Code, and an .mcp.json pointing at the hosted HTTP endpoint https://prompt-lab-mcp.up.railway.app/mcp. Provider keys are never stored in the MCP config: export ANTHROPIC_API_KEY, GEMINI_API_KEY or OPENAI_API_KEY into the agent's environment and it registers them per session, with the provider detected from the key prefix. Self-hosting needs UPSTASH_REDIS_REST_URL, UPSTASH_REDIS_REST_TOKEN and PROMPT_LAB_UI_URL. MIT licensed.
One command plus a key — npx -y mcp-remote https://prompt-lab-mcp.up.railway.app/mcp, then supply credentials