Summary
A second opinion on your model's answer, one property at a time.
Instead of a single opinionated quality score, each property is its own call — is this grounded in the context, does it leak personal data, is the tone right — so you can gate on the one that matters for your use and ignore the rest.
What it is
An evaluation server where each tool scores one property of a model response — how well it is grounded in its context, what it leaks, how it reads — against Trustwise's metrics.
What you get
- Grounding checks: faithfulness_metric, context_relevancy_metric, answer_relevancy_metric, adherence_metric
- Safety checks: toxicity_metric, pii_metric, prompt_injection_metric, sensitivity_metric, refusal_metric
- Readability and voice: clarity_metric, simplicity_metric, formality_metric, tone_metric, helpfulness_metric
- Summary quality: summarization_metric and completion_metric
- stability_metric, for how consistent the answer is
- The same tools inside a pipeline or an orchestration framework as in a chat
Requirements
A Trustwise API key as TW_API_KEY, and Docker — it runs from the ghcr.io/trustwiseai/trustwise-mcp-server image. TW_BASE_URL points it at a specific Trustwise instance instead of the default.
Setup effort
One command plus a key — docker run -i --rm -e TW_API_KEY ghcr.io/trustwiseai/trustwise-mcp-server:latest, then supply credentials