<p align="center">
    <img src="https://raw.githubusercontent.com/shayaShav/flatten-mcp/main/assets/logo.png" alt="flatten-mcp logo" width="160">
</p>

# flatten-mcp

<p align="center">
  <a href="https://www.npmjs.com/package/flatten-mcp"><img alt="npm version" src="https://img.shields.io/npm/v/flatten-mcp.svg"></a>
  <a href="https://www.npmjs.com/package/flatten-mcp"><img alt="npm downloads" src="https://img.shields.io/npm/dm/flatten-mcp.svg"></a>
  <a href="https://github.com/shayaShav/flatten-mcp/actions/workflows/test.yml"><img alt="tests" src="https://github.com/shayaShav/flatten-mcp/actions/workflows/test.yml/badge.svg"></a>
  <a href="https://github.com/shayaShav/flatten-mcp/blob/main/LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-green.svg"></a>
  <a href="https://modelcontextprotocol.io"><img alt="Model Context Protocol" src="https://img.shields.io/badge/MCP-server-6E56CF.svg"></a>
  <a href="https://docs.claude.com/en/docs/claude-code"><img alt="Built for Claude Code" src="https://img.shields.io/badge/built%20for-Claude%20Code-D97757.svg"></a>
  <a href="#openai-codex-cli-too"><img alt="Codex CLI supported" src="https://img.shields.io/badge/Codex%20CLI-supported-10a37f.svg"></a>
  <a href="https://smithery.ai/servers/shaya-shaviv/flatten-mcp"><img alt="Smithery" src="https://img.shields.io/badge/Smithery-listed-FF5601.svg"></a>
</p>

<p align="center"><sub>Claude Code · OpenAI Codex CLI — macOS · Linux · Windows · WSL2</sub></p>

> **The same Claude Code session, resumed for a fraction of the tokens — nothing summarized,
> nothing lost.** A long session carries its entire history on every message, and deep in,
> most of that is no longer conversation: it is files Claude already read and boiled down to
> a line or two, still riding along in context. flatten-mcp does what the name says — the
> digested bulk moves to a local backup, a small marker stays behind, and every prompt,
> reply, and decision survives word for word, any block one call from coming back. Claude
> gets cheaper and sharper.

<p align="center">
  <img src="https://raw.githubusercontent.com/shayaShav/flatten-mcp/main/assets/flatten-hero.png" alt="One flatten takes the same Claude Code session from 340,071 tokens to 132,800; the report shows 61% saved and /resume loads the lighter copy" width="820">
</p>
<p align="center"><sub>One flatten: <code>340,071 → 132,800</code> tokens — the same session, <code>/resume</code>'d back 61% lighter, every word of the conversation intact.</sub></p>

What is all that weight? The 2 MB log that became one line, the screenshot Claude
described, the five files it summarized — raw source that did its job and turned into a
sentence. flatten-mcp moves each tool result above a size threshold into a local backup next
to the session and leaves a small `[FLATTENED …]` marker in its place; any block is one call
from coming back.

|  | `/compact` | Auto tool-result clearing | **flatten-mcp** |
| --- | --- | --- | --- |
| What happens | history rewritten into a summary | old tool results cleared as the limit nears | bulk moved to a local backup, markers remain |
| Speed & cost | slow — a full model pass over your history, spends tokens/budget | automatic, no token cost | **instant, zero tokens — a local file rewrite** |
| Lossy? | yes — an interpretation | cleared content is gone from context | no — byte-identical restore any time |
| You choose when? | you or the auto-cliff | automatic | yes |
| Session file on disk | rewritten | unchanged | shrinks; the backup keeps every original |

**Try it first — nothing installed, nothing written:**

```bash
npx -y flatten-mcp-session flatten --dry-run
```

Run it from a project you use Claude Code in — any terminal, or straight from inside a
session with the `!` prefix: it prints what a flatten would save on your most recent
session and writes nothing.

The report doesn't just total the savings — it shows exactly where the weight sits:

```
  context tokens   : 168707 saved of 444927 (37.9%) exact
  breakdown        : (exact tokens)
    claude-in-chrome (MCP)  126x  8.9 MB    94,342 tok
    Read                    55x   1.6 MB    64,028 tok
    Bash                    23x   40.9 KB   7,301 tok
    …
```

Add `--breakdown deep` and every row opens into its individual blocks — each command,
file, and screenshot that built the bulk, largest first.

## Quick start

Runs through `npx` — no global install, nothing added to your project. Every read/write
stays inside the agent's own session store — Claude Code's `~/.claude/projects/` by
default, Codex's `~/.codex/sessions` when you target it — and there are zero network
calls by default. (Node ≥ 18, which Claude Code already runs on.)

**1. Install** — either path:

```bash
# Terminal: register the server user-wide (pinned; use @latest if you prefer auto-updates)
claude mcp add flatten -s user -- npx -y flatten-mcp@2.6.0

# Optional: the /flatten slash command
curl -fsSL https://raw.githubusercontent.com/shayaShav/flatten-mcp/main/commands/flatten.md -o ~/.claude/commands/flatten.md
```

```bash
# Or as a Claude Code plugin — registers the server AND bundles /flatten in one step
claude plugin marketplace add shayaShav/flatten-mcp
claude plugin install flatten-mcp@flatten-mcp
```

**2. Restart Claude Code (or open a new session)** — an already-open session does not pick
up a newly added server. Check with `/mcp`: `flatten` should be listed as connected.

**3. Use it — two steps, always:**

```
/flatten     → the session file is rewritten in place, right after a complete backup is written
/resume      → switch to another session and back; the reloaded copy is the lighter one
```

Until you `/resume`, the window you are in still holds the full pre-flatten copy in memory —
nothing will look different. After it, watch the context indicator drop.

<p align="center">
  <img src="https://raw.githubusercontent.com/shayaShav/flatten-mcp/main/assets/flatten-demo.gif" alt="The full run captured live: /flatten reports the savings, /resume loads the lighter session at 132,800 tokens" width="820">
</p>
<p align="center"><sub>The full run, captured live: <code>/flatten</code> reports the savings, <code>/resume</code> loads the lighter copy.</sub></p>

<details>
<summary><b>Manual registration / from source</b></summary>

In `~/.claude.json` or your project's `.mcp.json`:

```json
{
    "mcpServers": {
        "flatten": { "command": "npx", "args": ["-y", "flatten-mcp@2.6.0"] }
    }
}
```

For development: `git clone https://github.com/shayaShav/flatten-mcp.git && cd flatten-mcp
&& npm install`, then point the config at `node /absolute/path/to/dist/index.js`.

</details>

## Usage

- Bare `/flatten` (or asking *"flatten this session"*) targets the **current** session — the
  server identifies it from `CLAUDE_CODE_SESSION_ID`. Pass a UUID to target another session.
- Preview first with a **dry run** — *"dry-run flatten this session"* — nothing is written.
- See **what's in there**: every dry run includes the breakdown; pass `breakdown: true` on a
  real flatten (CLI: `--breakdown`) to get it there too, or `breakdown: "deep"`
  (CLI: `--breakdown deep`) to list every block — each command and file with its own size
  and tokens, largest first.
- Undo completely by asking to **unflatten**: every block returns to its exact original value.
- Don't flatten a session that is mid-generation; flatten between turns, or from a second
  window — which also keeps the tool schemas out of your working session entirely.
- Working in **Codex CLI** instead? The same tools take `agent: "codex"` — see
  [OpenAI Codex CLI, too](#openai-codex-cli-too).

> [!TIP]
> Flattening is pure file surgery — no model intelligence involved — so a fast, inexpensive
> model (`/model haiku`) flattens just as well as a frontier one. Or skip the model turn
> entirely: `! npx -y flatten-mcp-session flatten` runs the CLI from inside the session —
> zero tokens, no model pass over your still-heavy history.

## What you'll actually save

The reduction is the bulk you remove, not a fixed percentage:

- **Read-heavy sessions** (large files, long logs, screenshots): the demo above measured
  **340,071 → 132,800 tokens, a 61% cut**. The more ingested bulk, the bigger the cut —
  base64-screenshot-heavy sessions can go higher.
- **Prose-heavy sessions** (little external data): savings are small — there's not much
  bulk to move.

A common point to reach for it is around **200k** tokens; the most dramatic cuts show up
at 250k–400k. It's repeatable — a re-flatten only touches bulk that arrived since the last
one. The three tool schemas cost **~1,200 tokens per turn** while the server is connected;
one flatten of a read-heavy session removes orders of magnitude more from every later turn
(207k in the demo), and the separate-window pattern above makes even that overhead zero.

Where the saving lands: a session re-sends its history on every message, so removed bulk
stops costing you on every later turn — prompt caching softens that (cache reads bill at
roughly a tenth of the input price), but a cold cache re-warms the full context at a
premium, and on subscription plans the same tokens draw down your usage limits. Just as
important, a leaner window keeps the context ceiling — and lossy auto-compaction — further
away.

## OpenAI Codex CLI, too

The same engine, the same guarantees, for Codex sessions. Install flatten-mcp as a Codex
plugin — it bundles the server plus a `flatten-codex` skill that keeps every call on the
Codex store — or skip installing and drive the terminal CLI:

```bash
# Codex plugin
codex plugin marketplace add https://github.com/shayaShav/flatten-mcp
# then /plugins → install flatten-mcp

# Or zero-install: the CLI reaches Codex rollouts directly
npx -y flatten-mcp-session flatten --agent codex
codex resume <session-id>        # the resumed rollout is the lighter copy
```

Measured live on real sessions: resuming a flattened rollout sent **39–51% fewer input
tokens** by Codex's own counter (`27,029 → 13,139` on the largest run). Codex rebuilds
resume context from the rollout file, so what leaves the file leaves the model's context —
and restore stays byte-identical, proven on every candidate-bearing rollout of a 728-file
corpus (209/209). A resumed Codex session has no MCP retrieve tool, so each marker's tail
carries the exact command that fetches its block back
(`flatten-mcp-session retrieve <session> <id> --agent codex`); the backup sits next to the
rollout, invisible to `codex resume`.

Two honest notes. Codex token savings are local estimates — Anthropic's `count_tokens`
does not measure GPT-5's tokenizer, so the context total comes from Codex's own usage
events. And the size of the win depends on how much output Codex persisted: it sometimes
stores only a preview of large command output. Claude Code stays the first-class path;
Codex rides the `SessionAdapter` seam more agents can follow (see
[Compatibility & roadmap](#compatibility--roadmap)).

## Tools

| Tool | What it does |
| --- | --- |
| `flatten_session` | Move bulky tool results into the backup, leaving `[FLATTENED …]` markers. Crash-safe, reversible. No argument = current session; supports `dry_run`, `breakdown` (`true` or `"deep"`), `min_size`, `include_tool_use_result`, and `agent` (`"claude"` default, `"codex"`). |
| `retrieve_flattened` | Fetch one original block back by id — text, or a flattened screenshot re-rendered as a real image. Takes `agent` to match the session's store. |
| `unflatten_session` | Reverse everything: re-inline every block from the backup, then delete the backup. Takes `agent` too. |

In a flattened session the model sees markers like this, carrying everything needed to fetch
the original:

```
[FLATTENED id=toolu_01AbC… tool=Read file_path=/src/server.ts | text 48213B/612L | session=2f9c… | retrieve_flattened(id,session) for raw content]
```

## How it works

- **One backup, not deletion.** `<session>.jsonl.bak` holds the complete session fully
  inlined; the live file carries markers. Kept in lockstep every run
  (`backup = unflatten(live)`, `live = flatten(backup)`).
- **Crash-safe.** Originals are written to the backup *before* bulk leaves the session,
  each write via atomic temp-file-and-rename — an interrupted run can't leave a
  half-written session.
- **Self-cleaning.** A full unflatten restores everything inline and deletes the backup —
  zero artifacts left.
- **Re-flatten friendly.** As the session grows, run it again; only new bulk is touched,
  and content added after a flatten is never lost on restore.
- **Lossless.** Text and base64 images are stored exactly as they appeared —
  `unflatten_session` restores byte-identical values.
- **Honest numbers.** Claude Code stores each tool result twice on disk but sends one to
  the model; reports separate `diskBytesSaved` from `contextTokensSaved` (the number that
  matters), estimated locally — or exact via `count_tokens` when you opt in with
  `FLATTEN_COUNT_EXACT=1` (plus `ANTHROPIC_API_KEY`). The breakdown's per-group counts
  ride the same opt-in; per-block figures in deep mode stay estimates.

Details — session JSONL format, backup model, marker protocol — in
[docs/ARCHITECTURE.md](https://github.com/shayaShav/flatten-mcp/blob/main/docs/ARCHITECTURE.md).

**Validate the claims yourself:** (1) pick a meaty session; (2) ask for a dry run and read
the report; (3) `/flatten` for real, `/resume`, and watch the context indicator drop by the
reported amount; (4) unflatten and confirm the session file returns byte-identical (diff
against a copy if you kept one).

## Security & verification

- **Provenance you can check.** Every release is published from CI via npm **trusted
  publishing (OIDC)** with **provenance attestations**, from a **signed tag** — no npm
  token exists anywhere. Verify: `npm audit signatures`. Pin an exact version (as the
  Quick start does) and the committed `package-lock.json` documents the tree we test
  against; `npx` resolves the two direct dependencies' own trees at install time — audit
  with `npm ls --omit=dev`.
- **File access.** Confined to the session stores:
  `<CLAUDE_CONFIG_DIR or ~/.claude>/projects/<encoded-project-dir>/` and, under
  `agent: "codex"`, `<CODEX_HOME or ~/.codex>/sessions/` — rewriting session
  files there is the tool's entire job, always backup-first and atomic. The one exception:
  `flatten-mcp-session retrieve --out` writes a retrieved image where you tell it to.
- **Network.** Zero outbound calls unless you explicitly opt in to exact token counts.
  With **both** `FLATTEN_COUNT_EXACT=1` and `ANTHROPIC_API_KEY` set — key presence alone
  is not enough — exactly one endpoint is ever contacted:
  `POST api.anthropic.com/v1/messages/count_tokens` (free). The request body contains the
  counting model id (`FLATTEN_COUNT_MODEL`) and a single user message holding the tool
  results being flattened, reduced to their text and image blocks; a second identical call
  counts the replacement markers, and — when the breakdown is requested — one more of the
  same shape per report row. Sent only to Anthropic; the key is read from the
  environment and never stored or logged. There is no other outbound URL in the codebase.
  The optional `flatten-mcp-http` bin (below) accepts *inbound* connections when you run
  it — localhost by default — and makes no outbound calls.
- **Small enough to audit in one sitting.** A few small TypeScript files, two direct
  dependencies, no telemetry, no shell, no hooks — no analytics, no spawned processes, no
  permission bypasses. Vulnerability reports: [SECURITY.md](SECURITY.md).

## Beyond Claude Code — CLI & library

The same engine ships as a terminal CLI, an in-memory library, and a Streamable HTTP
server, so raw Messages API callers (any language) get the identical flatten/unflatten
semantics with no MCP and no session files.

<details>
<summary><b><code>flatten-mcp-session</code> — flatten Claude Code and Codex CLI sessions from the terminal (no LLM turn, zero tokens)</b></summary>

```bash
npx -y flatten-mcp-session flatten                     # most-recent session in this project
npx -y flatten-mcp-session flatten <session> --dry-run
npx -y flatten-mcp-session flatten last --dry-run --breakdown deep   # per-block x-ray
npx -y flatten-mcp-session list
npx -y flatten-mcp-session unflatten <session>
npx -y flatten-mcp-session retrieve <session> <tool_use_id> --out shot.png

# Target OpenAI Codex CLI rollouts instead of Claude Code sessions:
npx -y flatten-mcp-session flatten --agent codex
npx -y flatten-mcp-session list --agent codex
npx -y flatten-mcp-session retrieve <uuid> <call_id> --agent codex
```

- `<session>`: UUID, `last`, `"last N"`, `current`, or a keyword — same grammar as the MCP
  tool. Shared flags: `--agent <claude|codex>` (default `claude`), `--project-dir`,
  `--claude-dir`, `--codex-home`, `--json`.
- `--agent codex` targets `~/.codex/sessions` (override with `--codex-home` or `$CODEX_HOME`).
  A UUID reaches any rollout in the tree; `last`/`"last N"`/keyword are scoped to rollouts
  whose cwd equals the project dir, and `current` falls back to `last` (Codex has no
  live-session linkage). Codex token savings are reported as local estimates only — the
  context total comes from Codex's own usage events. The backup sits next to the rollout as
  `<rollout>.jsonl.bak`, invisible to `codex resume`.
- Drives the exact same on-disk engine as the MCP server — ideal for cron and scripts.
  After a real flatten, `/resume` the session in Claude Code (or `codex resume` the rollout)
  to load the lighter copy.

</details>

<details>
<summary><b><code>flatten-mcp-cli</code> — flatten a raw Messages API conversation over stdin/stdout</b></summary>

```bash
echo '[{"role":"user","content":"hi"}]' | npx -y flatten-mcp-cli --flatten
npx -y flatten-mcp-cli --flatten --min-size 2000 < body.json > flattened.json
npx -y flatten-mcp-cli --unflatten < flattened.json > restored.json
```

- `--flatten` prints `{ messages, extracted, flattenedCount, contextTokensSaved, … }` —
  **persist `extracted` yourself; you are the store.** `--unflatten` restores
  byte-for-byte. No server, no disk, no network. Bad input → stderr + exit 1.

</details>

<details>
<summary><b>Library API — <code>flattenMessages</code> / <code>unflattenMessages</code> in-memory</b></summary>

```ts
import { flattenMessages, unflattenMessages } from 'flatten-mcp';

const { messages, extracted, contextTokensSaved } = flattenMessages(myMessages);
// send `messages` to the API; persist `extracted` yourself — you are the store.
const original = unflattenMessages(messages, extracted);   // byte-for-byte restore
```

- Synchronous, never mutates input (deep-copies first). `flattenRequestBody` /
  `unflattenRequestBody` handle a full `{ system, messages, tools, … }` body.
- Exact token counts (optional, async): `flattenMessagesExact` uses Anthropic's free
  `count_tokens` when `ANTHROPIC_API_KEY` is set — calling the `*Exact` variant is the
  opt-in here (`countExact: false` forces the estimate); the `FLATTEN_COUNT_EXACT`
  variable gates only the MCP server and session CLI.
- **Prompt-caching caveat:** flattening earlier messages changes the cached prefix and
  invalidates `cache_control` breakpoints from that point on — flatten **before**
  establishing a breakpoint, or the cache re-write can cost more than the flatten saves
  in short-lived conversations.

</details>

<details>
<summary><b><code>flatten-mcp-http</code> — the in-memory engine over MCP Streamable HTTP</b></summary>

```bash
npx -y flatten-mcp-http            # POST http://127.0.0.1:8787/mcp
npx -y flatten-mcp-http --port 3000 --host 0.0.0.0
```

- Serves `flatten_messages` / `unflatten_messages` — the same stateless in-memory engine
  as the library, callable from any MCP client or hosted registry inspector. Persist the
  returned `extracted` yourself and feed it back to restore, exactly like the library.
- The three disk tools are **not** exposed over HTTP: they operate on the local Claude
  Code session store, which does not exist wherever a remote client calls from. (On the
  stdio server, `FLATTEN_INMEMORY_TOOLS=1` adds these two tools alongside the disk ones.)
- No auth, permissive CORS, **no outbound network calls** — the tools are pure functions
  over the request's JSON. Binds `127.0.0.1` by default; put your own proxy/auth in front
  before exposing it further. Note the transport cost: the conversation you flatten
  travels to this server and back — inside your own process, prefer the library.

</details>

<details>
<summary><b>Hosted endpoint — call the engine with nothing installed</b></summary>

A public `flatten-mcp-http` instance runs at `https://shaya.cloud/flatten-mcp`
(Streamable HTTP, no credentials). Same contract as the library: it serves
`flatten_messages` / `unflatten_messages` only — persist the returned `extracted`
yourself — and the disk tools still need the local install above. Mind the transport:
your conversation travels to this server and back, so send only what you would route
through a third-party service.

```bash
# Claude Code
claude mcp add --transport http flatten-remote https://shaya.cloud/flatten-mcp
```

- **Claude (claude.ai / Desktop)**: Settings → Connectors → Add custom connector → paste the URL.
- **Cursor**: [Add to Cursor](https://cursor.com/install-mcp?name=flatten&config=eyJ1cmwiOiJodHRwczovL3NoYXlhLmNsb3VkL2ZsYXR0ZW4tbWNwIn0=), or `"flatten": { "url": "https://shaya.cloud/flatten-mcp" }` in `mcp.json`.
- **VS Code**: `"flatten": { "type": "http", "url": "https://shaya.cloud/flatten-mcp" }` in `mcp.json`.
- Liveness: `curl https://shaya.cloud/flatten-mcp/health`

</details>

## FAQ

**Won't Anthropic just build this in?** Claude Code already clears old tool results
automatically near the limit (see the table up top). Flatten is a different contract:
*you* pick the moment, the restore is byte-identical, and the on-disk session you
`/resume` from actually shrinks.

**Will the model fetch a flattened block, or hallucinate around it?** Each marker carries
the id and session, and in practice the model calls `retrieve_flattened` when it needs raw
bytes back. Deterministic recovery is always there regardless: `unflatten_session`
re-inlines everything.

**Does it need Node in my project?** No — it runs through `npx` ephemerally and touches
only your agent's session files, not your project or toolchain.

**Can a team use it?** It's per-developer (each dev's local session store). Standardize by
committing the `mcpServers` block to your project's `.mcp.json`, or point the team at the
plugin install.

## Compatibility & roadmap

- **Claude Code and OpenAI Codex CLI session stores** — selected per call with
  `--agent <claude|codex>` (CLI) or the `agent` parameter (MCP), default `claude`. The disk
  engine runs behind a `SessionAdapter` seam, so each agent's store is a self-contained
  adapter over the shared block logic; more agents can follow the same seam — contributions
  welcome ([CONTRIBUTING.md](CONTRIBUTING.md)). **WSL2 counts as Linux**: if your agent runs
  inside WSL2, flatten-mcp runs in the same environment and targets those sessions normally.
  Native Windows is untested.

## Configuration

Operates on the project the CLI runs in; pass `project_dir` on any call to target another.

| Env var | Required | Purpose |
| --- | --- | --- |
| `CLAUDE_CONFIG_DIR` | no | Claude config dir whose `projects/` store is read (default `~/.claude`). Same variable Claude Code uses for profiles, so an alternate-profile server targets its own sessions automatically; override per call with `claude_dir`. |
| `CODEX_HOME` | no | Codex home whose `sessions/` tree is targeted under `agent: "codex"` (default `~/.codex`); override per call with `codex_home` (MCP) / `--codex-home` (CLI). |
| `FLATTEN_COUNT_EXACT` | no | Set to `1` to count token savings **exactly** via Anthropic's free `count_tokens` — the only outbound call, and it needs `ANTHROPIC_API_KEY` too. Off by default: key presence alone never triggers the request (see Security). |
| `ANTHROPIC_API_KEY` | no | The key for the exact count. Ignored by the MCP server and session CLI unless `FLATTEN_COUNT_EXACT=1`. |
| `FLATTEN_COUNT_MODEL` | no | Model id for the exact count (default `claude-haiku-4-5-20251001`). |
| `FLATTEN_INMEMORY_TOOLS` | no | Set to `1` to also register `flatten_messages`/`unflatten_messages` on the stdio server (see the HTTP section above). Off by default to keep the local tool surface lean. |

## Uninstall

Unflatten anything you want back inline **first** — a flattened session needs its
`<session>.jsonl.bak` for `retrieve_flattened`/`unflatten_session`, and uninstalling does
not remove backups. Then:

```bash
claude mcp remove flatten -s user && rm -f ~/.claude/commands/flatten.md   # terminal install
claude plugin uninstall flatten-mcp                                        # plugin install
```

To reclaim disk for sessions you'll never restore, delete their `.jsonl.bak` files from
`~/.claude/projects/<encoded-project-dir>/` (Claude Code) or from next to the rollouts
under `~/.codex/sessions/` (Codex).

## Contributing

Issues and PRs welcome — dev setup, project map, and workflow in
[CONTRIBUTING.md](CONTRIBUTING.md); security reports via [SECURITY.md](SECURITY.md).

## License

[MIT](LICENSE) © Shaya Shaviv