Labsco
fkiene logo

llmtrim

★ 139

from fkiene

MCP server that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history, tool output, and code before they're sent: -31% input / -74% output, measured live.

🔥🔥🔥🔥✓ VerifiedFreeAdvanced setup
#email#automation#social-media#database#cloud#devops#git#maps

llmtrim

llmtrim is a local proxy that compresses your LLM API requests so you pay less, with no change to the answers.** It sits between your AI tools and the provider, strips the wasted tokens out of every request, and forwards it on. You get the same answers for a smaller bill.

−31% input and −74% output tokens**, measured live across 112 A/B cases, with no change in answer quality.

One static binary. ~5 ms per call, no model to load.

Use it as a proxy, a CLI, an MCP server, or a library (Python · Ruby · Swift · Kotlin · JS · TS · WASM).

What it does • In action • Get started • CLI & library • Works with • Numbers • How it compares

What it actually does

You run Claude Code, Codex, Cursor, or your own app. Every time it talks to an LLM, it sends a big blob of text: your system prompt, the tool definitions, the whole conversation history, and the raw output of every command it ran. You pay for every one of those tokens, on every single turn.

A lot of that text is waste. A 200-line build log where only 2 lines are errors. A tool schema resent identically 50 times. A JSON array with 500 near-identical rows. The model doesn't need the bulk of it to answer well, but you're billed for all of it.

llmtrim removes the waste before it's sent. It installs as a local proxy that sits between your tool and the LLM provider. Requests pass through it, get compressed, and continue to the provider. The reply comes back unchanged. Your tool doesn't know it's there; you just get a smaller bill.

Copy & paste — that's it
 before: your tool ───── full request ─────▶ OpenAI / Anthropic / …
 ◀──────── reply ──────────

 after: your tool ──▶ llmtrim ──smaller──▶ OpenAI / Anthropic / …
 (on your machine)
 ◀──────── reply ────────── (same answer)

** [!IMPORTANT] It can never make your bill bigger or break a request. Every compression step is re-measured with the provider's real tokenizer; if a step doesn't actually save tokens, it's reverted. If the provider rejects the compressed request, the original is resent verbatim. Worst case is zero savings, never a worse outcome.

Everything runs locally. Nothing is ever sent to us.

See it on real output

Here's one real thing llmtrim does, end to end. An AI agent ran a build, and the bash tool returned a 58-line log. Only two lines matter (the errors), but all 58 get sent to the model and billed.

Before , what the model would receive (58 lines, 4,662 chars):

Copy & paste — that's it
[2026-06-13T10:02:00Z] INFO compiling module core::worker::task_0 (incremental)
[2026-06-13T10:02:01Z] INFO compiling module core::worker::task_1 (incremental)
[2026-06-13T10:02:02Z] INFO compiling module core::worker::task_2 (incremental)
... 27 more near-identical INFO lines ...
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
... 25 more INFO lines ...
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO build failed, 2 errors

After , what llmtrim sends instead (5 lines, 978 chars, −79% ):

Copy & paste — that's it
[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO build failed, 2 errors

Both errors and the summary survive verbatim . The repetitive INFO lines fold into a template plus their values, losslessly, because the range is regular (task_0..task_29). The model still sees exactly what happened; it just costs a fifth as much.

If that's useful to you, a ⭐ helps other people find it.

Try it yourself on any request body:

Copy & paste — that's it
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log-folding is just one of ten compressors. A different one re-encodes bulky JSON arrays into a compact table, with the same data in a third of the tokens:

Copy & paste — that's it
before: [{"id":1,"city":"Paris","ok":true},{"id":2,"city":"Lyon","ok":false}, … 200 rows]
after: [200]{id,city,ok}: 1,Paris,true; 2,Lyon,false; … (TOON encoding, lossless)

Each compressor fires only where it pays:

Where the waste is What llmtrim does Tool output (build logs, diffs, grep dumps, big JSON) Keep the signal (errors, changes, matches), fold the noise Long context (pasted docs, history) Rank and keep the chunks relevant to the question; drop the rest Source code Keep the bodies of relevant functions, reduce the rest to signatures Tool schemas (resent every turn) Trim descriptions, drop unused tools, keep the cache prefix stable JSON / record arrays Re-encode to a compact table format, sample huge arrays The model's reply Ask for terser output where it won't hurt the answer

Full stage reference (all 10 compressors)** Stages run in savings order. Nothing under a cache_control marker is ever rewritten.

Stage What it does When it runs tool-output Lossless template fold first, then window logs · diffs · grep · dumps down to errors / changes / matches tool results cache discipline Mark + stabilize the invariant prefix (sort tools/schema · OpenAI prompt_cache_key) so it stays cached tools lexical retrieval BM25+ ranking with RM3 feedback · TextTiling topic cuts · budgeted non-redundant selection; question protected long context skeletonization tree-sitter keeps relevant function bodies, drops the rest to signatures (14 languages) code serialize + hygiene Minify JSON, encode record arrays to TOON or CSV, Unicode-normalize always · lossless json sample Down-sample huge record arrays: first/last + outliers + a query-biased diverse sample big JSON dedup Collapse duplicate + near-duplicate lines (prose only) always output control Terse instruction · Chain-of-Draft · token budget · native JSON schema auto tool layer Static tool selection + description trimming tools multimodal Downscale images to the provider's resolution cap images

Default auto switches each stage on only where it pays. safe runs the lossless stages only. Full config →

Get started

** [!NOTE] Works with any tool that routes through HTTPS_PROXY: Claude Code, Codex, Cursor, Aider, your own app. GitHub Copilot pins its certificates and can't be intercepted ( full list ).

Copy & paste — that's it
# 1. Install (any OS, prebuilt binary, no Rust needed)
npm install -g @llmtrim/cli@latest && llmtrim setup

# 2. Open a new shell. Your AI tools now route through llmtrim automatically.

# 3. Watch the savings add up as you work
llmtrim status

No Node? Use an installer instead:

Copy & paste — that's it
# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

Or your own package manager, same binary everywhere: brew install fkiene/tap/llmtrim · cargo binstall llmtrim · scoop install llmtrim · docker run ghcr.io/fkiene/llmtrim. Full options in INSTALL.md.

Is this safe to install?

setup is a local HTTPS proxy, the same technique as mitmproxy, scoped to LLM APIs. It changes exactly three things (a CA certificate in ~/.llmtrim/, a proxy setting in your shell profile, a login service), and llmtrim uninstall reverses all three. No API keys are stored (it forwards your tool's own auth), and your prompts never touch disk; only an anonymous count of tokens saved is kept. Full threat model: SECURITY.md.

What gets installed, and how to verify the cert is harmless**

  • A private certificate in ~/.llmtrim/, cryptographically constrained to LLM API domains. It cannot read your bank, email, or any other traffic, even if the key were stolen. Check that yourself:
Copy & paste — that's it
llmtrim ca # prints the cert path
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"
# those domains are the only ones it can ever touch

  • A proxy setting in your shell profile (HTTPS_PROXY + NODE_EXTRA_CA_CERTS).

  • A background service that starts at login.

If the service stops, your tools fail fast with a connection error rather than silently bypassing it.

Day-to-day commands

Copy & paste — that's it
llmtrim status # health + savings dashboard (aliases: monitor, gain)
llmtrim doctor # something off? end-to-end diagnosis; each check names its fix
llmtrim start # start the background proxy
llmtrim stop # stop it
llmtrim serve # run in the foreground instead (Ctrl-C to quit)
llmtrim wrap claude # run an agent, guaranteeing this session routes through llmtrim (fails fast if it can't)
llmtrim update # update to the latest release + restart
llmtrim uninstall # exact inverse of setup: removes all three changes

llmtrim status --daily (or --weekly / --monthly) gives a time-series report; add --json or --csv to export.

Use it as a CLI, MCP, or library

The same compression runs with no proxy and no setup, as a one-shot CLI, an embeddable Rust crate, native bindings for Python, Ruby, Swift and Kotlin, or a WebAssembly module for JavaScript (browser, Node, Cloudflare Worker). No extra model calls, no network: the deterministic engine runs in your process.

Language Install Rust cargo add llmtrim-core Python pip install llmtrim Ruby gem install llmtrim Kotlin implementation("io.github.fkiene:llmtrim:0.5.0") (Maven Central) Swift .package(url: "https://github.com/fkiene/llmtrim-swift", from: "0.1.8") (SwiftPM)

CLI. Pipe a request in, get a compressed one out:

Copy & paste — that's it
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send --provider openai # compress, call, print

Rust. The engine is the llmtrim-core crate (no tokio, no network in its dependency tree):

Copy & paste — that's it
use llmtrim_core::{compress, compress_with_config, config::DenseConfig, ir::ProviderKind};

// None auto-detects the provider from the request shape.
let out = compress(request_json, Some(ProviderKind::OpenAi))?;
println!("{} -> {} input tokens", out.input_tokens_before, out.input_tokens_after);

// …or pass an explicit preset/config:
let out = compress_with_config(request_json, Some(ProviderKind::OpenAi), &DenseConfig::preset("agent").unwrap())?;

Python / Ruby / Swift / Kotlin. One flat compress(input, provider, preset) call, generated from the same Rust engine via UniFFI. The compiled engine is bundled in each package, so there's no Rust toolchain to install:

Copy & paste — that's it
import llmtrim

out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")
print(out.input_tokens_before, "->", out.input_tokens_after)

** [!NOTE] Every binding returns the compressed request_json plus the before/after token counts, and maps errors to native exceptions. Per-language install and usage live in crates/llmtrim-uniffi.

JavaScript / TypeScript (WebAssembly). The same compress(input, provider, preset) call, compiled to WebAssembly, runs in the browser, Node, Bun, Deno, or a Cloudflare Worker with no network or filesystem access. The output type is fully typed in TypeScript:

Copy & paste — that's it
import { compress } from "@llmtrim/js"; // @llmtrim/wasm is an alias for the same package

const out = compress(requestJson, "openai", "aggressive");
console.log(out.input_tokens_before, "->", out.input_tokens_after);

[!NOTE] To stay small, the WASM build uses the estimate tokenizer: the absolute token counts are approximate, but the savings percentage is unaffected. Build and usage live in crates/llmtrim-wasm.

MCP server. llmtrim mcp speaks the Model Context Protocol over stdin/stdout, so any MCP client can compress payloads and read your savings without the proxy. It exposes three tools: llmtrim_compress (compress a full request body, honoring your ~/.llmtrim config like the proxy), llmtrim_compress_text (shrink one text blob, lossless), and llmtrim_stats (your savings ledger). Every call records to the same ledger, so MCP traffic shows up in llmtrim status.

Copy & paste — that's it
llmtrim mcp install # register with Claude Code (one command)
llmtrim mcp install --print # or print the config to paste into any other client

The printed block is the standard MCP config; for a client you edit by hand it looks like:

Copy & paste — that's it
{
 "mcpServers": {
 "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
 }
}

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA. That covers all 18 agents below plus any HTTPS_PROXY-aware CLI or SDK you build yourself:

Tool Works Notes Claude Code ✅ Prompt-cache discount stays intact Codex CLI ✅ Gemini CLI ✅ Cursor (IDE), Cline, Roo, Kilo Code ✅ VS Code extensions; set NODE_EXTRA_CA_CERTS for the Node host process Goose, OpenCode, Crush, Mux, Forge, OpenClaw, Pi/OMP ✅ CLI agents on standard provider hosts Qwen Code, Grok CLI, Kimi Code, Mistral Vibe ✅ Provider hosts ship in the llm_providers registry, intercepted out of the box Aider, any other HTTPS_PROXY-aware CLI ✅ Hermes, Droid (BYOK mode) ✅ Interceptable only when a direct provider key is configured; see guide for Hermes Your own app / SDK ✅ Or call the CLI / library directly GitHub Copilot ❌ Certificate pinning blocks interception Warp, Devin ❌ Provider call is server-side; a local proxy never sees it Cursor Agent, Kiro ❌ Routes through a vendor gateway, not a standard provider host

Prefer no proxy? Any MCP client (Claude Code, Cursor, custom agents) can call llmtrim directly as tools instead: run llmtrim mcp install, or see CLI, MCP, or library .

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Every non-LLM connection passes through untouched.

The numbers

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

original compressed saved input tokens 71,031 49,062 −31% output tokens 25,843 6,628 −74% round-trip cost $0.0365 $0.0126 −66% answer quality 78.9% 82.2% no measured degradation

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

Accuracy preserved on standard benchmarks The same A/B on the standard academic suites, at a conservative shape-matched preset (qwen3-next-80b, paired 95% CI). Quality is the score on the original request vs the compressed one. GSM8K comes from the frontier above (n=12); the other three are the named benchmarks readers compare against (n=20 each):

benchmark task scorer input saved quality (orig → comp) retention GSM8K grade-school math numeric-exact −47%¹ 100% → 92% −8pp TruthfulQA (MC1) factual truthfulness choice-exact 0% 75% → 75% +0.0±0.0pp SQuAD v2 extractive QA token-F1 / EM 11% 84% → 84% −0.0±15.2pp BFCL (live_multiple) function calling tool-call match 33% 95% → 95% +0.0±15.2pp

Three rows compress with no quality loss; GSM8K is the one dip:

  • BFCL drops the tool schemas the query doesn't need (a menu of 2 to 37 candidates per call).

  • SQuAD v2 still answers its unanswerable questions correctly.

  • TruthfulQA holds factual accuracy exactly: its ~75-token prompts are almost all answer text, so the safe preset finds nothing to cut.

  • GSM8K trades −8pp of accuracy for −71% cost, so measure per workload before enabling its reasoning preset. ¹Its input goes negative because that preset injects a Chain-of-Draft instruction whose payoff is output-side (see the frontier table).

Evidence and a one-line reproduce (named-benchmark snapshot):

Copy & paste — that's it
make -C crates/llmtrim-cli/bench data
(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \
 --corpus bench/data/squad2.jsonl --preset rag \
 --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Methodology, per-corpus frontier, and confidence intervals: crates/llmtrim-cli/bench/README.md. Reproduce it:

Copy & paste — that's it
make -C crates/llmtrim-cli/bench data # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)
(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite) # live A/B across all corpora (needs OPENROUTER_API_KEY)
(cd crates/llmtrim-cli/bench/scripts && PYTHONPATH=. python3 -m benchkit.tools.chart) # regenerate the chart + table

How it compares

Each tool compresses one slice of the request. llmtrim compresses input and output, leaves the cached prefix untouched to keep the prompt cache stable, and scores on whether the answer survives the cut, not on tokens removed. Both axes below use the o200k_base encoder and reproduce from this repo.

llmtrim Headroom RTK caveman Compresses input · output input tool/CLI output model output Skips no-op transforms ✅ ❌ ❌ n/a One static binary ✅ Python + models ✅ ✅

Input

Input reduction (deterministic) next to answer quality from a live A/B. Quality is the drop vs llmtrim at each tool's compared setting (✅ held, a statistical tie; ❌ significantly lower), so a big reduction with a ❌ means the tool bought tokens by losing answers:

Tool Reduction Quality vs llmtrim Overhead llmtrim auto 25% ✅ ref ~5 ms llmtrim aggressive 28% ✅ ref ~5 ms Headroom (ML on) 24% ✅ tie ~0.9 s leanctx / LLMLingua-2 52-81% ❌ 18% lower ~6 s entroly 80-89% ❌ 42% lower <1 ms

Overhead is the median per-call compress time (Python wall-clock, not like-for-like CPU): Headroom and leanctx run ML on CPU here (faster on a GPU) and pay a one-time model load on top (~3 s and ~4 s); llmtrim is Rust and entroly is lexical, so neither does.

  • auto is the quality-gated default; aggressive accepts lossy cuts where the gate holds.

  • Headroom drops to 0% with its ML disabled (its routers no-op on prose).

  • leanctx and entroly are lossy with no quality gate; entroly has no low-reduction mode.

Headroom ties at matched reduction (24-25%, n=30, not significant) but its longer answers hit the model's output-token limit and get truncated 12 times to llmtrim's 2, the output inflation behind its higher cost. leanctx (measured at 26%) and entroly (at 69%, its mildest) score significantly lower than llmtrim (n=20), and fall further at their headline reductions (vs-leanctx, vs-entroly).

Output

Output reduction by asking for terser responses, on a paid live call over 9 coding prompts:

output cut overhead / request caveman 80% 949 tokens llmtrim output_terse 69% 19 tokens

The cost is the 949-token system prompt caveman resends on every request (right column); llmtrim's is 19 for nearly the same cut. Both still net-save here, so caveman's deeper cut comes out ahead only when the output it removes is worth more than the 949 tokens it adds back (vs-caveman artifact).

The tools stack: RTK shrinks CLI output, then llmtrim compresses the tool schemas on top. Full head-to-heads: crates/llmtrim-cli/bench/README.md.

Known limits

These are surfaced by the same A/B that proves the savings:

  • Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.

  • Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.

  • The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.

  • "Lossless" is input-side, not response restoration. A lossless stage preserves the information the model reads (a folded log run, a TOON-encoded array, an abbreviation legend the model decodes in-prompt), and the token gate reverts any input cut that doesn't pay off. The engine does not transform the model's response back to an original form.

Acknowledgments

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Papers + crates behind each stage Retrieval & context: BM25 (Robertson & Zaragoza 2009, bm25); BM25+ (Lv & Zhai, CIKM 2011); RM3 (Lavrenko & Croft, SIGIR 2001); TextTiling (Hearst, CL 1997); TextRank (Mihalcea & Tarau, EMNLP 2004); MMR (Carbonell & Goldstein, SIGIR 1998); Submodular objective (Lin & Bilmes, ACL 2011); modified-greedy knapsack maximizer (Tang et al., SIGMETRICS 2021, arXiv:2008.05391); DPP diverse sampling (Chen et al., NeurIPS 2018); Lost in the Middle (arXiv:2307.03172); DSLR (arXiv:2407.03627).

Code: RepoCoder (arXiv:2303.12570); Hierarchical Context Pruning (arXiv:2406.18294); The Hidden Cost of Readability (arXiv:2508.13666); Minification token accounting (arXiv:2606.01326).

Tool output: Drain (He et al., ICWS 2017); Brain (Yu et al., IEEE TSC 2023); LogLSHD (arXiv:2504.02172).

Dedup & abbreviation: SimHash (Charikar, STOC 2002, gaoya); CompactPrompt (arXiv:2510.18043); Maximal repeats (arXiv:1304.0528) + Re-Pair (Larsson & Moffat, DCC 1999).

Output control: Chain-of-Draft (arXiv:2502.18600); TALE (arXiv:2412.18547).

Serialization: TOON (Token-Oriented Object Notation), Johann Schopplich.

Built on tiktoken-rs, tree-sitter, image, whatlang, hudsucker, rusqlite, and more.

Found a problem?

Run llmtrim doctor for an end-to-end diagnosis; each failing check names its fix. Found a request it mangled? Set LLMTRIM_CAPTURE_DIR and open an issue with the before/after capture, since a repro is a fix. And if it saved you money, a ⭐ helps others find it.

Star history

Licensed under MPL-2.0. Use llmtrim freely in your stack, including commercially, with no source-disclosure obligation for your own code; the file-level copyleft applies only to modifications you make to llmtrim's own source files. Contributions via DCO sign-off.