Labsco
MCP SERVER

Pull verifiable, self-contained factual claims out of a piece of prose.

LLM Evaluation & Observability
Summary
An independent implementation of a published method, and it says so.

The paper's prompts were modified to work with structured outputs, and the author states outright that this is not the official implementation. What it gives you that a summariser does not is decontextualisation: each claim stands on its own, which is the property that makes checking it — by the bundled prompt or by a person — possible at all.

What it is

A local Python server implementing the Claimify method from the paper "Towards Effective Extraction and Evaluation of Factual Claims" — a four-stage pipeline that turns text into atomic factual claims.

What you get
  • Extraction in four stages: split into sentences with their surrounding context, keep only the sentences carrying verifiable propositions, resolve ambiguity or discard what cannot be clarified, then decompose into atomic self-contained claims
  • Opinions and speculation filtered out at the selection stage instead of being extracted as facts
  • OpenAI structured outputs throughout, so results come back typed rather than as prose to parse
  • Each extraction stored as MCP resources — an aggregate at `claim://extraction_<n>_<timestamp>` and one resource per claim at `claim://<slug>`
  • A `verify_claim` prompt that has the model check a single claim against authoritative sources and answer VERIFIED, UNCERTAIN or DISPUTED, with at least three referenced sources when verified
  • The original language preserved, and logging of every model call and pipeline stage
Requirements

Python 3.10+ and an OpenAI API key in `OPENAI_API_KEY`, with a model that supports structured outputs — `gpt-4o` or the cheaper `gpt-4o-mini`. Create a virtualenv, `pip install -r requirements.txt`; the NLTK punkt data downloads on first run. Point your client at the virtualenv's Python and `claimify_server.py`. `LOG_LLM_CALLS`, `LOG_OUTPUT` and `LOG_FILE` control the logging.