LabscoMCP early access ↗
AdamGustavssonMCP SERVER

Claimify

Pull verifiable, self-contained factual claims out of a piece of prose.

by AdamGustavsson ↗LLM Evaluation & ObservabilityCommunity

LABSCO SUMMARY

An independent implementation of a published method, and it says so.

The paper's prompts were modified to work with structured outputs, and the author states outright that this is not the official implementation. What it gives you that a summariser does not is decontextualisation: each claim stands on its own, which is the property that makes checking it — by the bundled prompt or by a person — possible at all.

What it is

A local Python server implementing the Claimify method from the paper "Towards Effective Extraction and Evaluation of Factual Claims" — a four-stage pipeline that turns text into atomic factual claims.

What you get

  • Extraction in four stages: split into sentences with their surrounding context, keep only the sentences carrying verifiable propositions, resolve ambiguity or discard what cannot be clarified, then decompose into atomic self-contained claims
  • Opinions and speculation filtered out at the selection stage instead of being extracted as facts
  • OpenAI structured outputs throughout, so results come back typed rather than as prose to parse
  • Each extraction stored as MCP resources — an aggregate at claim://extraction_<n>_<timestamp> and one resource per claim at claim://<slug>
  • A verify_claim prompt that has the model check a single claim against authoritative sources and answer VERIFIED, UNCERTAIN or DISPUTED, with at least three referenced sources when verified
  • The original language preserved, and logging of every model call and pipeline stage

Requirements

Python 3.10+ and an OpenAI API key in OPENAI_API_KEY, with a model that supports structured outputs — gpt-4o or the cheaper gpt-4o-mini. Create a virtualenv, pip install -r requirements.txt; the NLTK punkt data downloads on first run. Point your client at the virtualenv's Python and claimify_server.py. LOG_LLM_CALLS, LOG_OUTPUT and LOG_FILE control the logging.

Setup effort

Build from source — git clone https://github.com/AdamGustavsson/ClaimsMCP