Labsco
shayaShav logo

flatten-mcp

★ 3

from shayaShav

Flatten Claude Code sessions: keep every prompt and event verbatim, resume at a lower token count.

🔥🔥🔥✓ VerifiedPaid serviceNeeds API keys
#email#automation#analytics#payments#git#security

flatten-mcp logo

flatten-mcp

Move the bulk out of a Claude Code session — the huge file reads, logs, and screenshots the model already digested — into a local backup, fully reversibly. On Pro/Max that means compaction fires later and the model stays sharp; on API billing it means paying for fewer tokens. Every prompt and event stays verbatim.

npm downloads

340,071 → 132,800 tokens — a 61% lighter session, every line of history intact.
macOS · Linux · WSL2 — native Windows untested

Demo: a 340,071-token Claude Code session is flattened, /resume'd, and comes back 61% lighter at 132,800 tokens — history verbatim

Most of a long session's tokens are raw source the model already distilled into prose — the 2 MB log it boiled down to one line, the screenshot it described, the five files it summarized. flatten-mcp moves that bulk into a local backup next to the session and leaves a small [FLATTENED …] marker in its place. Nothing is rewritten, nothing is summarized away; any block is one call from coming back. The whole thing is a handful of small TypeScript files with two direct dependencies — small enough to audit in one sitting.

/compactAuto tool-result clearingflatten-mcp
What happenshistory rewritten into a summaryold tool results cleared as the limit nearsbulk moved to a local backup, markers remain
Lossy?yes — an interpretationcleared content is gone from contextno — byte-identical restore any time
You choose when?you or the auto-cliffautomaticyes
Session file on diskrewrittenunchangedshrinks; the backup keeps every original

Taste it first — nothing installed, nothing written:

npx -y flatten-mcp-session flatten --dry-run

Run it from a project you use Claude Code in: it prints the exact savings a flatten would give your most recent session and writes nothing.

What you'll actually save

The reduction is the bulk you remove, not a fixed percentage:

  • Read-heavy sessions (large files, long logs, screenshots): the demo above measured 340,071 → 132,800 tokens, a 61% cut. The more ingested bulk, the bigger the cut — base64-screenshot-heavy sessions can go higher.
  • Prose-heavy sessions (little external data): savings are small — there's not much bulk to move.

A common point to reach for it is around 200k tokens; the most dramatic cuts show up at 250k–400k. It's repeatable — a re-flatten only touches bulk that arrived since the last one. The three tool schemas cost ~1,200 tokens per turn while the server is connected; one flatten of a read-heavy session removes orders of magnitude more from every later turn (207k in the demo), and the separate-window pattern above makes even that overhead zero.

Tools

ToolWhat it does
flatten_sessionMove bulky tool results into the backup, leaving [FLATTENED …] markers. Crash-safe, reversible. No argument = current session; supports dry_run, min_size, include_tool_use_result.
retrieve_flattenedFetch one original block back by id — text, or a flattened screenshot re-rendered as a real image.
unflatten_sessionReverse everything: re-inline every block from the backup, then delete the backup.

In a flattened session the model sees markers like this, carrying everything needed to fetch the original:

[FLATTENED id=toolu_01AbC… tool=Read file_path=/src/server.ts | text 48213B/612L | session=2f9c… | retrieve_flattened(id,session) for raw content]

How it works

  • One backup, not deletion. <session>.jsonl.bak holds the complete session fully inlined; the live file carries markers. Kept in lockstep every run (backup = unflatten(live), live = flatten(backup)).
  • Crash-safe. Originals are written to the backup before bulk leaves the session, each write via atomic temp-file-and-rename — an interrupted run can't leave a half-written session.
  • Self-cleaning. A full unflatten restores everything inline and deletes the backup — zero artifacts left.
  • Re-flatten friendly. As the session grows, run it again; only new bulk is touched, and content added after a flatten is never lost on restore.
  • Lossless. Text and base64 images are stored exactly as they appeared — unflatten_session restores byte-identical values.
  • Honest numbers. Claude Code stores each tool result twice on disk but sends one to the model; reports separate diskBytesSaved from contextTokensSaved (the number that matters), estimated locally — or exact via count_tokens when you opt in with FLATTEN_COUNT_EXACT=1 (plus ANTHROPIC_API_KEY).

Details — session JSONL format, backup model, marker protocol — in docs/ARCHITECTURE.md.

Validate the claims yourself: (1) pick a meaty session; (2) ask for a dry run and read the report; (3) /flatten for real, /resume, and watch the context indicator drop by the reported amount; (4) diff <session>.jsonl.bak against a pre-flatten copy if you kept one, then unflatten and confirm the restore is byte-identical.

Security & verification

  • Provenance you can check. Every release is published from CI via npm trusted publishing (OIDC) with provenance attestations, from a signed tag — no npm token exists anywhere. Verify: npm audit signatures. Pin an exact version (as the Quick start does) and the committed package-lock.json documents the tree we test against; npx resolves the two direct dependencies' own trees at install time — audit with npm ls --omit=dev.
  • File access. Confined to the session store, <CLAUDE_CONFIG_DIR or ~/.claude>/projects/<encoded-project-dir>/ — rewriting session files there is the tool's entire job, always backup-first and atomic. The one exception: flatten-mcp-session retrieve --out writes a retrieved image where you tell it to.
  • Network. Zero outbound calls unless you explicitly opt in to exact token counts. With both FLATTEN_COUNT_EXACT=1 and ANTHROPIC_API_KEY set — key presence alone is not enough — exactly one endpoint is ever contacted: POST api.anthropic.com/v1/messages/count_tokens (free). The request body contains the counting model id (FLATTEN_COUNT_MODEL) and a single user message holding the tool results being flattened, reduced to their text and image blocks; a second identical call counts the replacement markers. Sent only to Anthropic; the key is read from the environment and never stored or logged. There is no other outbound URL in the codebase. The optional flatten-mcp-http bin (below) accepts inbound connections when you run it — localhost by default — and makes no outbound calls.
  • No telemetry, no shell, no hooks. No analytics, no spawned processes, no permission bypasses. Vulnerability reports: SECURITY.md.

Beyond Claude Code — CLI & library

The same engine ships as a terminal CLI, an in-memory library, and a Streamable HTTP server, so raw Messages API callers (any language) get the identical flatten/unflatten semantics with no MCP and no session files.

flatten-mcp-session — flatten Claude Code sessions from the terminal (no LLM turn, zero tokens) 
npx -y flatten-mcp-session flatten                     # most-recent session in this project
npx -y flatten-mcp-session flatten <session> --dry-run
npx -y flatten-mcp-session list
npx -y flatten-mcp-session unflatten <session>
npx -y flatten-mcp-session retrieve <session> <tool_use_id> --out shot.png
  • <session>: UUID, last, "last N", current, or a keyword — same grammar as the MCP tool. Shared flags: --project-dir, --claude-dir, --json.
  • Drives the exact same on-disk engine as the MCP server — ideal for cron and scripts. After a real flatten, /resume the session in Claude Code to load the lighter copy.
flatten-mcp-cli — flatten a raw Messages API conversation over stdin/stdout 
echo '[{"role":"user","content":"hi"}]' | npx -y flatten-mcp-cli --flatten
npx -y flatten-mcp-cli --flatten --min-size 2000 < body.json > flattened.json
npx -y flatten-mcp-cli --unflatten < flattened.json > restored.json
  • --flatten prints { messages, extracted, flattenedCount, contextTokensSaved, … } — persist extracted yourself; you are the store. --unflatten restores byte-for-byte. No server, no disk, no network. Bad input → stderr + exit 1.
Library API — flattenMessages / unflattenMessages in-memory 
import { flattenMessages, unflattenMessages } from 'flatten-mcp';

const { messages, extracted, contextTokensSaved } = flattenMessages(myMessages);
// send `messages` to the API; persist `extracted` yourself — you are the store.
const original = unflattenMessages(messages, extracted);   // byte-for-byte restore
  • Synchronous, never mutates input (deep-copies first). flattenRequestBody / unflattenRequestBody handle a full { system, messages, tools, … } body.
  • Exact token counts (optional, async): flattenMessagesExact uses Anthropic's free count_tokens when ANTHROPIC_API_KEY is set — calling the *Exact variant is the opt-in here (countExact: false forces the estimate); the FLATTEN_COUNT_EXACT variable gates only the MCP server and session CLI.
  • Prompt-caching caveat: flattening earlier messages changes the cached prefix and invalidates cache_control breakpoints from that point on — flatten before establishing a breakpoint, or the cache re-write can cost more than the flatten saves in short-lived conversations.
flatten-mcp-http — the in-memory engine over MCP Streamable HTTP 
npx -y flatten-mcp-http            # POST http://127.0.0.1:8787/mcp
npx -y flatten-mcp-http --port 3000 --host 0.0.0.0
  • Serves flatten_messages / unflatten_messages — the same stateless in-memory engine as the library, callable from any MCP client or hosted registry inspector. Persist the returned extracted yourself and feed it back to restore, exactly like the library.
  • The three disk tools are not exposed over HTTP: they operate on the local Claude Code session store, which does not exist wherever a remote client calls from. (On the stdio server, FLATTEN_INMEMORY_TOOLS=1 adds these two tools alongside the disk ones.)
  • No auth, permissive CORS, no outbound network calls — the tools are pure functions over the request's JSON. Binds 127.0.0.1 by default; put your own proxy/auth in front before exposing it further. Note the transport cost: the conversation you flatten travels to this server and back — inside your own process, prefer the library.

Compatibility & roadmap

  • Claude Code's session store only, for now — the paths and JSONL schema are specific to it. WSL2 counts as Linux: if your Claude Code runs inside WSL2, flatten-mcp runs in the same environment and targets those sessions normally. Native Windows is untested.
  • The CLI and library above are the first adapter over the shared block logic; porting to other agents means abstracting the storage seam — contributions welcome (CONTRIBUTING.md).