# mcp-promptdiff

**Zero-dependency stdio MCP server and CLI for token-aware prompt version diffs.**

> Compare two prompt strings or files and get a machine-readable token delta report — no external APIs, no native dependencies, runs anywhere.

[![PyPI version](https://img.shields.io/pypi/v/mcp-promptdiff.svg)](https://pypi.org/project/mcp-promptdiff/)
[![MIT license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

## Quick Start

```bash
pip install git+https://github.com/prasad-a-abhishek/mcp-promptdiff.git
```

```python
from mcp_promptdiff import diff_prompts, estimate_tokens

# Token-aware diff between two prompts
report = diff_prompts(
    "You are a helpful assistant.",
    "You are a helpful, concise assistant.",
)
print(f"Token delta: {report.token_delta:+d}")   # e.g. +4
print(f"Changed: {report.changed}")              # True

# Estimate token count for any string
est = estimate_tokens("Hello 🤖 world")
print(f"Tokens: {est.token_count}")             # e.g. 5
print(f"Confidence: {est.confidence.value}")     # low (emoji)
```

## Performance & Benchmarks

Token estimation is a fast heuristic (byte-length ÷ 4), making it suitable for CI pipelines and high-throughput MCP tooling without network latency or native dependencies.

| Operation | Latency | Notes |
|---|---|---|
| `estimate_tokens` (1 KB ASCII) | < 1 ms | Pure Python, no I/O |
| `diff_prompts` (1 KB each) | < 2 ms | Section-level breakdown included |
| MCP server round-trip | < 5 ms | stdio, no network |

Local benchmark reproduction:

```bash
python3 benchmarks/run_benchmark.py
```

## Why mcp-promptdiff?

**The problem:** LLM application developers version-control prompts but get no token-level signal from `git diff`. You can't answer "how many tokens were added?" or "which section grew most?" from a character-level diff.

**Existing tools are too heavy:** `tiktoken` and `tokenizers` require native data files and per-model downloads. Git diff has no concept of token boundaries. The MCP ecosystem had no dedicated prompt diff tool.

**mcp-promptdiff is the minimal building block:** Zero dependencies, pure stdlib, runs in any Python 3.11+ environment. Estimates are clearly labeled approximate with confidence/method fields so callers decide how much to trust them.

## Key Features

- **Zero dependencies** — pure Python stdlib, no pip installs beyond the package itself
- **Both CLI and library API** — use in CI, code review, or embedded MCP tooling
- **MCP stdio server** — drop into any MCP client (Claude Desktop, Cursor, etc.)
- **Batch diff** — compare many prompt pairs in one call
- **Encoding presets** — `cl100k_approx`, `o200k_base`, `p50k_base`
- **Confidence labels** — every estimate reports LOW/MEDIUM/HIGH confidence so clients can decide trust level

## API Reference

### `estimate_tokens(text, encoding="cl100k_approx")`

Returns a `TokenEstimate`:

```python
@dataclass(frozen=True)
class TokenEstimate:
    text: str
    token_count: int         # estimated token count
    confidence: Confidence   # HIGH | MEDIUM | LOW
    method: str              # e.g. "cl100k_approx_byte_ratio"
    encoding: str            # the encoding used
```

### `diff_prompts(prompt_a, prompt_b, encoding="cl100k_approx")`

Returns a `TokenReport`:

```python
@dataclass(frozen=True)
class TokenReport:
    token_delta: int                     # tokens added (negative = removed)
    token_count_a: int
    token_count_b: int
    section_deltas: List[SectionDelta]   # per-paragraph breakdown
    changed: bool                        # True if content differs
    encoding: str
    method: str
    estimated_cost_a: float              # USD at ~$0.01/1k tokens
    estimated_cost_b: float             # USD at ~$0.01/1k tokens
```

### `diff_files(path_a, path_b, encoding="cl100k_approx")`

Same shape as `diff_prompts`, loaded from disk paths.

### `batch_diff(pairs, encoding="cl100k_approx")`

Takes a list of `PromptPair(prompt_a, prompt_b, label=None)` and returns a list of `TokenReport`.

### MCP Server

```python
# Run as stdio MCP server (stays alive on stdin/stdout)
from mcp_promptdiff.mcp_server import run
run()
```

Tools exposed: `diff_prompts`, `diff_files`, `estimate_tokens`, `batch_diff`

## CLI Reference

```
mcp-promptdiff diff --a "original" --b "revised"
mcp-promptdiff diff --file-a prompt_v1.md --file-b prompt_v2.md
mcp-promptdiff diff --file-a a.md --file-b b.md --format json
mcp-promptdiff estimate --text "Hello world"
mcp-promptdiff estimate --text "Hello 🤖" --encoding o200k_base
```

| Flag | Description |
|---|---|
| `--a`, `--b` | Prompt strings to diff |
| `--file-a`, `--file-b` | Paths to prompt files |
| `--encoding` | `cl100k_approx` (default), `o200k_base`, `p50k_base` |
| `--format` | `text` (default) or `json` |
| `--text` | Text string to estimate |

## Limitations

- **Estimates, not exact counts.** This is a byte-length heuristic (byte_length ÷ 4), not exact tokenization. Every result includes a `confidence` field so callers can decide how much to trust it.
- **No multi-turn conversation diffs.** Compares two flat prompt strings or two files only.
- **No HTTP/SSE transport.** stdio JSON-RPC only — the MCP server reads stdin and writes stdout.
- **No prompt optimization or templating.** This is a diff tool, not a prompt engineering tool.

## Non-Goals

- Exact tokenizer parity with any specific provider (OpenAI, Anthropic, etc.)
- Calling any hosted LLM API for evaluation
- Prompt optimization, variable substitution, or templating
- Persistent state or history
- HTTP/SSE transport

## License

MIT License — Abhishek Prasad