# PromptThrift MCP: Smart Token Compression for LLM Apps

> Cut 70-90% of your LLM API costs with intelligent conversation compression.
> Now with **Gemma 4 local compression**: smarter summaries, zero API cost.

<a href="https://glama.ai/mcp/servers/@woling-dev/promptthrift-mcp">
  <img width="380" height="200" src="https://glama.ai/mcp/servers/@woling-dev/promptthrift-mcp/badge" alt="PromptThrift MCP server" />
</a>

[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://python.org)
[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-purple.svg)](https://modelcontextprotocol.io)
[![Gemma 4](https://img.shields.io/badge/Gemma_4-Supported-orange.svg)](https://deepmind.google/models/gemma/gemma-4/)

⭐ **If this saves you money, star this repo!** ⭐

## The Problem

Every LLM API call resends your **entire conversation history**. A 20-turn chat costs 6x more per call than a 3-turn one, so you're paying for the same old messages over and over.

```
Turn 1:  ████ 700 tokens ($0.002)
Turn 5:  ████████████████ 4,300 tokens ($0.013)
Turn 20: ████████████████████████████████████████ 12,500 tokens ($0.038)
                                              ↑ You're paying for THIS every call
```

## The Solution

PromptThrift is an MCP server with 4 tools to slash your API costs:

| Tool | What it does | Impact |
|------|-------------|--------|
| `promptthrift_compress_history` | Compress old turns into a smart summary | 50-90% fewer input tokens |
| `promptthrift_count_tokens` | Track token usage & costs across 14 models | Know where money goes |
| `promptthrift_suggest_model` | Recommend cheapest model for the task | 60-80% on simple tasks |
| `promptthrift_pin_facts` | Pin critical facts that survive compression | Never lose key context |

## Why PromptThrift?

| | PromptThrift | Context Mode | Headroom |
|---|---|---|---|
| License | **MIT** (commercial OK) | ELv2 (no competing) | Apache 2.0 |
| Compression type | **Conversation memory** | Tool schema virtualization | Tool output |
| Local LLM support | **Gemma 4 via Ollama** | No | No |
| Cost tracking | **Multi-model comparison** | No | No |
| Model routing | **Built-in** | No | No |
| Pinned facts | **Never-Compress List** | No | No |

## Quick Start

### Install

**Option A: pip install (recommended)**
```bash
pip install git+https://github.com/woling-dev/promptthrift-mcp.git
```

**Option B: clone and install**
```bash
git clone https://github.com/woling-dev/promptthrift-mcp.git
cd promptthrift-mcp
pip install -e .
```

### Optional: Enable Gemma 4 Compression

For smarter AI-powered compression (free, runs locally):

```bash
# Install Ollama: https://ollama.com
ollama pull gemma4:e4b
```

PromptThrift auto-detects Ollama. If running → uses Gemma 4 for compression. If not → falls back to fast heuristic compression. Zero config needed.

### Claude Desktop

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "promptthrift": {
      "command": "python",
      "args": ["/path/to/promptthrift-mcp/server.py"]
    }
  }
}
```

### Cursor / Windsurf

Add to your MCP settings:

```json
{
  "mcpServers": {
    "promptthrift": {
      "command": "python",
      "args": ["/path/to/promptthrift-mcp/server.py"]
    }
  }
}
```

## Real-World Example

An AI coding assistant debugging a complex issue over 30+ turns:

**Before compression (sent every API call):**
```
User: My Next.js app throws a hydration error on the /dashboard page.
Asst: That usually means server and client HTML don't match. Can you share the component?
User: [pastes 50 lines of DashboardLayout.tsx]
Asst: I see the issue, you're using `new Date()` directly in render, which differs
      between server and client. Let me also check your data fetching...
User: I also get a warning about useEffect running twice.
Asst: That's React 18 Strict Mode. Not related to hydration. Let me trace the real bug...
User: Wait, there's also a flash of unstyled content on first load.
Asst: That's a separate CSS loading order issue. Let me address both...
      [... 25 more turns of debugging, trying fixes, checking logs ...]
User: OK it's fixed now! But I want to add dark mode next.
Asst: Great! For dark mode with Next.js + Tailwind, here are three approaches...
```
~8,500 tokens after 30 turns, **and growing every single API call**

**After Gemma 4 compression:**
```
[Compressed history]
Resolved Next.js hydration error in DashboardLayout.tsx caused by
Date() in render (fixed with useEffect). Unrelated: React 18 Strict Mode
double-fire (expected), CSS flash (fixed via loading order).
User now wants to add dark mode to Next.js + Tailwind app.
[End compressed history]

[Recent turns preserved, last 4 turns intact]
```
~1,200 tokens. **86% saved on every subsequent call**

**Cost impact at scale (Claude Sonnet @ $3/MTok):**
| Scenario | Without PromptThrift | With PromptThrift | Monthly Savings |
|----------|---------------------|-------------------|-----------------|
| 1 dev, 20 sessions/day | $5.10/mo | $0.72/mo | **$4.38** |
| Team of 10 devs | $51/mo | $7.20/mo | **$43.80** |
| Customer service bot (500 chats/day) | $255/mo | $36/mo | **$219** |
| AI agent platform (5K sessions/day) | $2,550/mo | $357/mo | **$2,193** |

## Pinned Facts (Never-Compress List)

Some facts must **never** be lost during compression: user names, critical preferences, key decisions. Pin them:

```
You: "Pin the fact that this customer is allergic to nuts"

→ promptthrift_pin_facts(action="add", facts=["Customer is allergic to nuts"])
→ This fact will appear in ALL future compressed summaries, guaranteed.
```

## Supported Models (April 2026 pricing)

| Model | Input $/MTok | Output $/MTok | Local? |
|-------|-------------|---------------|--------|
| gemma-4-e2b | **$0.00** | **$0.00** | Ollama |
| gemma-4-e4b | **$0.00** | **$0.00** | Ollama |
| gemma-4-27b | **$0.00** | **$0.00** | Ollama |
| gemini-2.0-flash | $0.10 | $0.40 | |
| gpt-4.1-nano | $0.10 | $0.40 | |
| gpt-4o-mini | $0.15 | $0.60 | |
| gemini-2.5-flash | $0.15 | $0.60 | |
| gpt-4.1-mini | $0.40 | $1.60 | |
| claude-haiku-4.5 | $1.00 | $5.00 | |
| gemini-2.5-pro | $1.25 | $10.00 | |
| gpt-4.1 | $2.00 | $8.00 | |
| gpt-4o | $2.50 | $10.00 | |
| claude-sonnet-4.6 | $3.00 | $15.00 | |
| claude-opus-4.6 | $5.00 | $25.00 | |

## How It Works

```
Before (every API call sends ALL of this):
┌──────────────────────────────────┐
│ System prompt      (500 tokens)  │
│ Turn 1: user+asst  (600 tokens)  │  ← Repeated every call
│ Turn 2: user+asst  (600 tokens)  │  ← Repeated every call
│ ...                              │
│ Turn 8: user+asst  (600 tokens)  │  ← Repeated every call
│ Turn 9: user+asst  (new)         │
│ Turn 10: user      (new)         │
└──────────────────────────────────┘
Total: ~6,500 tokens per call

After PromptThrift compression:
┌──────────────────────────────────┐
│ System prompt      (500 tokens)  │
│ [Pinned facts]      (50 tokens)  │  ← Always preserved
│ [Compressed summary](200 tokens) │  ← Turns 1-8 in 200 tokens!
│ Turn 9: user+asst  (kept)        │
│ Turn 10: user      (kept)        │
└──────────────────────────────────┘
Total: ~1,750 tokens per call (73% saved!)
```

### Compression Modes

| Mode | Method | Quality | Speed | Cost |
|------|--------|---------|-------|------|
| Heuristic | Rule-based extraction | Good (50-60% reduction) | Instant | Free |
| LLM (Gemma 4) | AI-powered understanding | Excellent (70-90% reduction) | ~10-15s | Free (local) |

PromptThrift automatically uses the best available method. Install Ollama + Gemma 4 for maximum compression quality.

### When Does Compression Shine?

Compression effectiveness scales with conversation length and redundancy:

| Conversation Length | Typical Reduction | Best For |
|---|---|---|
| Short (< 5 turns, mostly technical) | 15-25% | Minimal savings: keep as-is |
| Medium (10-20 turns, mixed chat) | 50-70% | Sweet spot: clear cost reduction |
| Long (30+ turns, debugging/iterating) | **70-90%** | Massive savings: compress early and often |

**Why?** Short, dense conversations have little filler to remove. Longer conversations accumulate greetings, repeated context, exploratory dead-ends, and verbose explanations, and those are exactly what the compressor strips away. A 30-turn debugging session with code snippets, back-and-forth troubleshooting, and final resolution compresses dramatically because only the conclusion and key decisions matter for future context.

**Rule of thumb:** Start compressing after 8-10 turns for best results.

## Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `PROMPTTHRIFT_OLLAMA_MODEL` | No | `gemma4:e4b` | Ollama model for LLM compression |
| `PROMPTTHRIFT_OLLAMA_URL` | No | `http://localhost:11434` | Ollama API endpoint |
| `PROMPTTHRIFT_DEFAULT_MODEL` | No | `claude-sonnet-4.6` | Default model for cost estimates |

## Security

- All data processed **locally** by default. Nothing leaves your machine
- Ollama compression runs 100% on your hardware
- **Post-compression sanitizer** strips prompt injection patterns from summaries
- API keys read from environment variables only, never hardcoded
- No persistent storage, no telemetry, no third-party calls

## Roadmap

- [x] Heuristic conversation compression
- [x] Multi-model token counting (14 models)
- [x] Intelligent model routing
- [x] **Gemma 4 local LLM compression via Ollama**
- [x] **Pinned facts (Never-Compress List)**
- [x] **Post-compression security sanitizer**
- [ ] Cloud-based compression (Anthropic/OpenAI API fallback)
- [ ] Prompt caching optimization advisor
- [ ] Web dashboard for usage analytics
- [ ] VS Code extension

## Contributing

PRs welcome! This project uses MIT license. Fork it, improve it, ship it.

## About BrandDefender.ai

**BrandDefender.ai** is the product line of **Wolin Global Media (沃嶺國際媒體)**, a Taiwan-based AI infrastructure studio helping brands get discovered, understood, and recommended by AI systems.

### What we build

🔍 **AEO Consulting (Answer Engine Optimization)**
Get your brand correctly cited by ChatGPT, Gemini, Perplexity, and Claude. We implement JSON-LD schema, optimize content structure, and monitor AI search presence for Taiwan food, tea, beauty, and lifestyle brands.
- Website: https://aibranddefender.com/
- Free AI brand scan: https://app.aibranddefender.com/

💬 **AI Customer Service (LINE Bot)**
Production-grade LINE chatbots with 3-layer memory, admin takeover, and Supabase backend. Already serving real brands in retail and F&B.
- Guide: [LINE AI Chatbot Guide](https://aibranddefender.com/blog-line-ai-chatbot-guide.html)

🧠 **AI Memory MCP Infrastructure**
Open-source MCP servers for Claude Code, Cursor, and LLM builders. Local-first, privacy-preserving, built to save API cost.
- **This repo** is one of them.
- Sibling tools: [promptforge](https://github.com/woling-dev/promptforge) · [promptthrift-mcp](https://github.com/woling-dev/promptthrift-mcp)

## Contact

- 📧 **Email**: service@wolinglobal.com
- 💬 **LINE**: [@886upktf](https://line.me/R/ti/p/@886upktf)
- 🌐 **Website**: https://aibranddefender.com/
- 🐙 **GitHub**: https://github.com/woling-dev

**台灣品牌想做 AEO audit**：我們提供 ChatGPT / Gemini / Perplexity 全面掃描 + JSON-LD 修補 + 月度監測。[Email](mailto:service@wolinglobal.com) 或 [LINE](https://line.me/R/ti/p/@886upktf) 直接找我們聊。

## License

[MIT License](LICENSE). Free for personal and commercial use.

---

© 2026 Wolin Global Media (沃嶺國際媒體).

**Star this repo if it saves you money!**