PromptThrift MCP: Smart Token Compression for LLM Apps

Cut 70-90% of your LLM API costs with intelligent conversation compression. Now with Gemma 4 local compression: smarter summaries, zero API cost.

⭐ If this saves you money, star this repo! ⭐

The Problem

Every LLM API call resends your entire conversation history. A 20-turn chat costs 6x more per call than a 3-turn one, so you're paying for the same old messages over and over.

Turn 1:  ████ 700 tokens ($0.002)
Turn 5:  ████████████████ 4,300 tokens ($0.013)
Turn 20: ████████████████████████████████████████ 12,500 tokens ($0.038)
                                              ↑ You're paying for THIS every call

The Solution

PromptThrift is an MCP server with 4 tools to slash your API costs:

Why PromptThrift?

Real-World Example

An AI coding assistant debugging a complex issue over 30+ turns:

Before compression (sent every API call):

User: My Next.js app throws a hydration error on the /dashboard page.
Asst: That usually means server and client HTML don't match. Can you share the component?
User: [pastes 50 lines of DashboardLayout.tsx]
Asst: I see the issue, you're using `new Date()` directly in render, which differs
      between server and client. Let me also check your data fetching...
User: I also get a warning about useEffect running twice.
Asst: That's React 18 Strict Mode. Not related to hydration. Let me trace the real bug...
User: Wait, there's also a flash of unstyled content on first load.
Asst: That's a separate CSS loading order issue. Let me address both...
      [... 25 more turns of debugging, trying fixes, checking logs ...]
User: OK it's fixed now! But I want to add dark mode next.
Asst: Great! For dark mode with Next.js + Tailwind, here are three approaches...

~8,500 tokens after 30 turns, and growing every single API call

After Gemma 4 compression:

[Compressed history]
Resolved Next.js hydration error in DashboardLayout.tsx caused by
Date() in render (fixed with useEffect). Unrelated: React 18 Strict Mode
double-fire (expected), CSS flash (fixed via loading order).
User now wants to add dark mode to Next.js + Tailwind app.
[End compressed history]

[Recent turns preserved, last 4 turns intact]

~1,200 tokens. 86% saved on every subsequent call

Cost impact at scale (Claude Sonnet @ $3/MTok):

Pinned Facts (Never-Compress List)

Some facts must never be lost during compression: user names, critical preferences, key decisions. Pin them:

You: "Pin the fact that this customer is allergic to nuts"

→ promptthrift_pin_facts(action="add", facts=["Customer is allergic to nuts"])
→ This fact will appear in ALL future compressed summaries, guaranteed.

Supported Models (April 2026 pricing)

How It Works

Before (every API call sends ALL of this):
┌──────────────────────────────────┐
│ System prompt      (500 tokens)  │
│ Turn 1: user+asst  (600 tokens)  │  ← Repeated every call
│ Turn 2: user+asst  (600 tokens)  │  ← Repeated every call
│ ...                              │
│ Turn 8: user+asst  (600 tokens)  │  ← Repeated every call
│ Turn 9: user+asst  (new)         │
│ Turn 10: user      (new)         │
└──────────────────────────────────┘
Total: ~6,500 tokens per call

After PromptThrift compression:
┌──────────────────────────────────┐
│ System prompt      (500 tokens)  │
│ [Pinned facts]      (50 tokens)  │  ← Always preserved
│ [Compressed summary](https://github.com/woling-dev/promptthrift-mcp/blob/HEAD/200 tokens) │  ← Turns 1-8 in 200 tokens!
│ Turn 9: user+asst  (kept)        │
│ Turn 10: user      (kept)        │
└──────────────────────────────────┘
Total: ~1,750 tokens per call (73% saved!)

Compression Modes

PromptThrift automatically uses the best available method. Install Ollama + Gemma 4 for maximum compression quality.

When Does Compression Shine?

Compression effectiveness scales with conversation length and redundancy:

Why? Short, dense conversations have little filler to remove. Longer conversations accumulate greetings, repeated context, exploratory dead-ends, and verbose explanations, and those are exactly what the compressor strips away. A 30-turn debugging session with code snippets, back-and-forth troubleshooting, and final resolution compresses dramatically because only the conclusion and key decisions matter for future context.

Rule of thumb: Start compressing after 8-10 turns for best results.

Environment Variables

Security

• All data processed locally by default. Nothing leaves your machine
• Ollama compression runs 100% on your hardware
• Post-compression sanitizer strips prompt injection patterns from summaries
• API keys read from environment variables only, never hardcoded
• No persistent storage, no telemetry, no third-party calls

Roadmap

• Heuristic conversation compression
• Multi-model token counting (14 models)
• Intelligent model routing
• Gemma 4 local LLM compression via Ollama
• Pinned facts (Never-Compress List)
• Post-compression security sanitizer
• Cloud-based compression (Anthropic/OpenAI API fallback)
• Prompt caching optimization advisor
• Web dashboard for usage analytics
• VS Code extension

About BrandDefender.ai

BrandDefender.ai is the product line of Wolin Global Media (沃嶺國際媒體), a Taiwan-based AI infrastructure studio helping brands get discovered, understood, and recommended by AI systems.

What we build

🔍 AEO Consulting (Answer Engine Optimization) Get your brand correctly cited by ChatGPT, Gemini, Perplexity, and Claude. We implement JSON-LD schema, optimize content structure, and monitor AI search presence for Taiwan food, tea, beauty, and lifestyle brands.

• Website: https://aibranddefender.com/
• Free AI brand scan: https://app.aibranddefender.com/

💬 AI Customer Service (LINE Bot) Production-grade LINE chatbots with 3-layer memory, admin takeover, and Supabase backend. Already serving real brands in retail and F&B.

• Guide: LINE AI Chatbot Guide

🧠 AI Memory MCP Infrastructure Open-source MCP servers for Claude Code, Cursor, and LLM builders. Local-first, privacy-preserving, built to save API cost.

• This repo is one of them.
• Sibling tools: promptforge · promptthrift-mcp

Contact

• 📧 Email: service@wolinglobal.com
• 💬 LINE: @886upktf
• 🌐 Website: https://aibranddefender.com/
• 🐙 GitHub: https://github.com/woling-dev

台灣品牌想做 AEO audit：我們提供 ChatGPT / Gemini / Perplexity 全面掃描 + JSON-LD 修補 + 月度監測。Email 或 LINE 直接找我們聊。

Quick Start

Install

Option A: pip install (recommended)

pip install git+https://github.com/woling-dev/promptthrift-mcp.git

Option B: clone and install

git clone https://github.com/woling-dev/promptthrift-mcp.git
cd promptthrift-mcp
pip install -e .

Optional: Enable Gemma 4 Compression

For smarter AI-powered compression (free, runs locally):

# Install Ollama: https://ollama.com
ollama pull gemma4:e4b

PromptThrift auto-detects Ollama. If running → uses Gemma 4 for compression. If not → falls back to fast heuristic compression. Zero config needed.

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "promptthrift": {
      "command": "python",
      "args": ["/path/to/promptthrift-mcp/server.py"]
    }
  }
}

Cursor / Windsurf

Add to your MCP settings:

{
  "mcpServers": {
    "promptthrift": {
      "command": "python",
      "args": ["/path/to/promptthrift-mcp/server.py"]
    }
  }
}

License

MIT License. Free for personal and commercial use.

© 2026 Wolin Global Media (沃嶺國際媒體).

Star this repo if it saves you money!