probe_allLABSCO SUMMARY
Ask the endpoint, not the status page.
A provider dashboard tells you whether an API is up; this tells you what your own calls are doing right now, against thresholds you chose. Because it is the same tool in a terminal, in CI and over MCP, an agent can confirm the model it is about to depend on is healthy before committing to the work.
What it is
A Go CLI for LLM serving reliability, with an MCP server built in. It sends a streaming request to each configured provider and model, then reports what users actually feel: how long before the first token arrives, how long the whole response takes, and whether the endpoint counts as healthy, degraded or erroring against thresholds you set.
What you get
- Probe every provider and model in
probes.ymlat once and read back TTFT, latency, throughput and health per model —probe_all - Probe one model with no config file at all, given a provider, a model name and the env var holding the key —
probe_model - List the providers and models already configured, with their thresholds, before deciding what to probe —
list_providers - Return the full parsed configuration including defaults, providers, models and thresholds —
get_config - OpenAI, Anthropic, Google, Azure OpenAI and AWS Bedrock are handled natively; anything OpenAI-compatible works by setting
base_url
Requirements
The llmprobe binary — a prebuilt release for Linux, macOS or Windows, or go install github.com/Jwrede/llmprobe@latest. Register it with claude mcp add --transport stdio llmprobe -- llmprobe mcp, or install the Claude Code plugin with claude plugin install Jwrede/llmprobe. The config-reading tools want a probes.yml holding providers, models and thresholds; API keys go in as ${ENV_VAR} references, so the file itself carries no secrets. It contacts only the endpoints you configure.
Setup effort
One command plus a key — go install github.com/Jwrede/llmprobe@latest, then supply credentials
USE CASES
01
Checking a provider before depending on it
Run a pass over every configured model and read back which are healthy, degraded or erroring.
02
Testing one model without touching config
Name the provider, the model and the env var holding its key, and get a single reading back.
03
Seeing what is configured before probing
List providers, models and their thresholds so the right target gets picked.
04
Explaining a slow answer
Separate time to first token from total latency to see whether the endpoint was slow to start or slow to finish.
4 tools
Listed in project docs. Your client may show a different list.
probe_modellist_providersget_configBEFORE YOU INSTALL
What to know first
Tools you get
4 tools named in the project's documentation.
Requires
- Go 1.25+ on PATH (for go install), or a prebuilt binary from the latest release
- A probes.yml listing the providers to probe, with the API key for each one you list (OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_OPENAI_API_KEY, AWS credentials for Bedrock, GROQ_API_KEY, DEEPSEEK_API_KEY or TOGETHER_API_KEY) in the environment; optional for local endpoints such as Ollama or vLLM, which take api_key: unused
INSTALLATION
Connect your client
Server setup
TERMINAL
go install github.com/Jwrede/llmprobe@latestMCP Clients
Claude Code — stdio
CLAUDE CODE — STDIO · TERMINAL
claude mcp add --transport stdio llmprobe -- llmprobe mcpThe configuration on this page does not set OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_OPENAI_API_KEY, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, GROQ_API_KEY, DEEPSEEK_API_KEY and TOGETHER_API_KEY, and only the providers you probe need one. You list providers in a probes.yml file whose credential fields reference environment variables (api_key: ${OPENAI_API_KEY}), so set each key in the environment llmprobe runs in.
Claude Code — plugin
CLAUDE CODE — PLUGIN · TERMINAL
claude plugin install Jwrede/llmprobeThe configuration on this page does not set OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_OPENAI_API_KEY, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, GROQ_API_KEY, DEEPSEEK_API_KEY and TOGETHER_API_KEY, and only the providers you probe need one. You list providers in a probes.yml file whose credential fields reference environment variables (api_key: ${OPENAI_API_KEY}), so set each key in the environment llmprobe runs in.
