faithfulness_metricLABSCO SUMMARY
A second opinion on your model's answer, one property at a time.
Instead of a single opinionated quality score, each property is its own call — is this grounded in the context, does it leak personal data, is the tone right — so you can gate on the one that matters for your use and ignore the rest.
What it is
An evaluation server where each tool scores one property of a model response — how well it is grounded in its context, what it leaks, how it reads — against Trustwise's metrics.
What you get
- Grounding checks: faithfulness_metric, context_relevancy_metric, answer_relevancy_metric, adherence_metric
- Safety checks: toxicity_metric, pii_metric, prompt_injection_metric, sensitivity_metric, refusal_metric
- Readability and voice: clarity_metric, simplicity_metric, formality_metric, tone_metric, helpfulness_metric
- Summary quality: summarization_metric and completion_metric
- stability_metric, for how consistent the answer is
- The same tools inside a pipeline or an orchestration framework as in a chat
Requirements
A Trustwise API key as TW_API_KEY, and Docker — it runs from the ghcr.io/trustwiseai/trustwise-mcp-server image. TW_BASE_URL points it at a specific Trustwise instance instead of the default.
Setup effort
One command plus a key — docker run -i --rm -e TW_API_KEY ghcr.io/trustwiseai/trustwise-mcp-server:latest
USE CASES
01
Checking a RAG answer is actually supported by its sources
Faithfulness scores the answer against the context it was given; context relevancy scores whether that context was worth retrieving.
02
Catching personal data before an answer is sent
Run the PII and sensitivity checks on the draft, not on the transcript afterwards.
03
Screening user input for prompt injection
Score the incoming text before it reaches the model that will act on it.
04
Getting the voice right for a customer-facing reply
Tone, formality and clarity are separate scores, so 'too stiff' and 'too dense' are different fixes.
05
Working out why an evaluation set regressed
Relevancy, adherence and stability separate a model that drifted off the question from one that stopped following instructions.
06
Judging a summary against the document it came from
Summarization and completion score coverage rather than fluency — the thing a reader notices only when something is missing.
19 tools
Listed in project docs. Your client may show a different list.
answer_relevancy_metriccontext_relevancy_metricpii_metricprompt_injection_metricsummarization_metricclarity_metricformality_metrichelpfulness_metricsensitivity_metricsimplicity_metrictone_metrictoxicity_metricrefusal_metriccompletion_metricadherence_metricstability_metriccarbon_metriccost_metricBEFORE YOU INSTALL
What to know first
Tools you get
19 tools named in the project's documentation.
Requires
- docker on PATH
- TW_API_KEY in your environment
Where it runs
Both documented clients run the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
docker run -i --rm -e TW_API_KEY ghcr.io/trustwiseai/trustwise-mcp-server:latestMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}
}Cursor
CURSOR · JSON
{
"mcpServers": {
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"-e",
"TW_BASE_URL",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}
}
