squeeze

Your context window is full of the same error repeated 200 times. squeeze cuts that noise — up to 70% on real compiler output, 45% on real failing test runs — and never deletes a byte.

A drop-in proxy that sits between your coding agent and the LLM API, compresses tool output before it reaches the context window, and keeps every original byte retrievable — by the model itself, over MCP. No code changes. No dependencies. One environment variable.

real tsc output:    25,166 chars  ->   7,517 chars   -70.1%
real pytest output: 20,110 chars  ->  11,040 chars   -45.1%

Measured on captured tool runs, not hand-written fixtures: scripts/collect-real.sh re-runs tsc, npm, pytest, node --test and this repository's own CI log, then the bench real command reports the honest numbers.

Contents

• Features

• Why

• Install

• Use

• Benchmark

• MCP

• Levels

• How it works

• Limits

• What it does not touch

• CLI

• Environment

• Tests

• Status

• Provenance

• License

Related MCP server: winnow

Features

• Lossless by design — squeeze never deletes. Every collapsed span is stored in a content-addressed ref store, so expand() reconstructs any document byte-for-byte.

• Reversible by the model — the squeeze_fetch MCP tool lets the agent pull the real text back on demand, instead of guessing what a ref meant.

• Drop-in — point *_BASE_URL at the proxy. No code, no config file, and zero runtime dependencies beyond Node itself.

• Honest — it compresses noise, never meaning. Source code and JSON are detected and left alone, and the benchmark reports 0.0% for them on purpose.

Why

Your context window is not full of your code. It is full of 200 identical TS2322 errors, 400 identical test-pass lines, and heartbeat logs that arrived at 09:00:11. The model reads all of it, every turn, and you pay for all of it.

Existing token compressors delete. If the model later needs the deleted lines, it cannot get them back — and you find out when it hallucinates a fix.

squeeze never deletes. It clusters, counts, and hands back a short ref:

src/mod0.ts(0,7): error TS2322: type mismatch  ⟨×119 more, ref=83a63083475a87f0⟩

That single line replaces 120. When the model decides it needs the real list, it calls squeeze_fetch and gets every original line back:

  83a63083475a87f0 ->  120 lines, 6,412 chars

Refs are content-addressed and persisted to disk, so they survive a proxy restart. A ref printed into a transcript last week still resolves today.

curl "http://127.0.0.1:8899/squeeze/expand?ref=83a63083475a87f0"

Install

Requires Node 22.18+. No runtime dependencies, no build step for you.

From npm:

npm install -g @arcnosixta/squeeze
squeeze doctor

The npm name squeeze belongs to someone else, so this package lives under its own scope. The tarball ships precompiled so the install works anywhere.

From the source (same thing, no install needed — Node runs the TypeScript directly):

git clone https://github.com/arcnosixta/squeeze
cd squeeze
node src/cli/main.ts doctor

Use

As a proxy (works with Claude Code, Codex, Cursor, anything speaking the Anthropic API or OpenAI's Chat Completions and Responses APIs):

node src/cli/main.ts serve

export ANTHROPIC_BASE_URL=http://127.0.0.1:8899
export OPENAI_BASE_URL=http://127.0.0.1:8899

On a file:

node src/cli/main.ts compress build.log --aggressive > small.log
node src/cli/main.ts inspect build.log --aggressive

Benchmark

Two sets of numbers, both reproducible on your machine.

Real tool output — files captured from actual runs, regenerated by scripts/collect-real.sh:

node src/cli/main.ts bench real aggressive

The byte total is dragged down by dpkg-log — 64% of the corpus, near-unique lines — while a plain average tells the other story: the artifacts that actually fill an agent's context, compiler errors and failing test runs, are where squeeze does its best work. The 0.0% row is part of the point: a passing test run is already noise, and squeeze leaves documents it cannot help alone.

Synthetic fixtures — the original hand-written corpus, kept for reference:

node src/cli/main.ts bench aggressive

The 0.0% rows are the point: json-dump and source-file are read line by line by the model, and squeeze leaves them alone rather than pretending otherwise. It compresses noise, never meaning.

Reproduce the end-to-end claim quickly:

node src/cli/main.ts selftest    # 9,779 -> 80 chars, ref verified byte-exact

MCP

Give the model a way to undo the compression on its own terms. This is the part that makes losslessness real: without it, a ref is a promise nobody can keep.

node src/cli/main.ts mcp

{
  "mcpServers": {
    "squeeze": {
      "command": "squeeze-mcp"
    }
  }
}

If you cloned instead of installing, point at the source file instead:

{
  "mcpServers": {
    "squeeze": {
      "command": "node",
      "args": ["/path/to/squeeze/src/mcp/server.ts"]
    }
  }
}

Two tools:

squeeze_fetch is the one that matters. The model reads ⟨×119 more, ref=83a6…⟩, decides the collapsed lines matter, and pulls them.

Levels

node src/cli/main.ts serve            # defaults to aggressive
SQUEEZE_LEVEL=safe node src/cli/main.ts serve

aggressive is the default: on real tool output safe saves almost nothing (no ANSI codes, few blank runs), while aggressive is where everything comes from. The two exceptions to reversibility are the same in both modes: ANSI codes and blank-line runs are discarded and keep no ref. Those bytes carry no information a model needs, but calling them "lossless" would be wrong. Every handle-based replacement — the ⟨ref=…⟩, ⟨×N more, ref=…⟩ and ⟨repeat of lines …; ref=…⟩ markers — stores its exact original span, and expand() resolves every one back in place, so aggregate output reconstructs byte-for-byte (the tests assert exactly that).

How it works

squeeze intercepts the request before it leaves the machine. Tool output is compressed in place, the original spans go into a content-addressed store, and the model sees a small prompt with short refs instead of a wall of repetition. When it needs the real lines, it calls squeeze_fetch and gets them back.

Five passes over each text block:

1. ANSI strip — colour codes carry no meaning to a model.

2. Template clustering — digit runs collapse to # regardless of neighbouring word characters, so suite1, module12 and (3,14) all match their siblings. Timestamps normalize for free, so adjacent log lines group.

3. Block dedup — runs of 8+ identical lines collapse to a range reference.

4. Document-wide dedup — real output rarely repeats adjacent lines: an error list names a different file and line number on every row. This pass keys on the whole document, so row9.ts(15,5) and row10.ts(16,5) collapse to the same handle even separated by other lines. Every line keeps its own slot — the original is swapped for a handle in place, never hoisted. Source code and JSON are detected and skipped, because there the repetition is the meaning.

5. Blank-run collapse — trailing newlines are preserved exactly, since tools diff on them.

Every collapse writes its original text into the store and emits a ref. Nothing clustering-related is unrecoverable: expand(input, store) replaces each handle with its stored span, in place.

Clustering happens in place. Hoisting log lines out of the transcript to group them better would squeeze a few percent more, but it would silently reorder the document — the model would read a reshuffled version of the user's history whose original sequence no longer exists anywhere. That is not lossless, so the tests enforce order preservation:

test/aggressive preserves document order when interleaving logs and content

A ref only ever holds a contiguous slice of the real input, and expand() reconstructs the original document byte-for-byte. Both properties are asserted, not assumed.

Limits

squeeze compresses the text payload of tool results. It does not:

• Touch images, audio, or binary tool results

• Scope the store per project — refs from two projects share one store, which is fine for expansion (keys are content hashes) but means no per-project eviction

• Guarantee savings on source code or JSON. Those are irreducible, and the benchmark reports 0.0% for them on purpose

If your workload is mostly cat-ing source files, this will not help you. If it is mostly test runs, compiler errors, and build logs, it will.

What it does not touch

• Your system prompt, instructions, or max_tokens

• Reasoning blocks in either spelling — Anthropic type: "thinking" / "redacted_thinking", Responses API "reasoning_text" / "summary_text" — because rewriting them changes reasoning semantics, so they pass through byte-identical

• tool_use_id, call_id and every structural field

• Your input object is cloned, never mutated

CLI

squeeze bench [safe|aggressive|none]   run the benchmark on synthetic fixtures
squeeze bench real [level]             benchmark real collected tool output
squeeze bench all [level]              synthetic and real side by side
squeeze serve                          start the proxy
squeeze compress <file|--level>        compress a file or stdin
squeeze expand <file|--level>          resolve every ref back to the original
squeeze inspect <file|--level>         preview changes, write nothing
squeeze doctor                         check environment and store
squeeze selftest                       verify compression and ref durability
squeeze mcp                            run the MCP server on stdio

--level accepts none, safe, aggressive, or the alias --aggressive. SQUEEZE_LEVEL sets the default when the flag is absent.

Environment

Endpoints: /squeeze/health, /squeeze/expand?ref=…

Tests

npm test                      # 31 unit tests
npm run test:proxy            # end-to-end proxy, real HTTP, all 3 levels
npm run test:mcp              # MCP over a real stdio JSON-RPC session
npm run test:persistence      # ref survives a full proxy restart

The unit suite covers the machinery directly: looksStructured protects JSON and source files, document-wide dedup collapses non-adjacent repeats while keeping every line's slot, and expand() reconstructs the original document byte-for-byte.

The proxy test asserts the full round trip: compress a nested Anthropic tool_result, confirm the forwarded payload shrank, confirm the untouched fields survived, then expand the ref and count all 120 original lines back.

The persistence test is the important one. It kills the proxy outright, starts a fresh one, and expands a ref created before the kill. That is the failure mode that would silently corrupt a long-running agent session.

Status

Working and tested: proxy, CLI, MCP server, persistent store, document-wide dedup, byte-exact expand(), and all three request shapes (Anthropic messages, OpenAI chat messages, OpenAI Responses input).

Not yet: per-project store scoping, and a proxy mode for deploying squeeze as a separate service rather than on the agent's own host.

Provenance

The real fixtures in fixtures/real/ and the terminal screenshot in the README are both generated, never hand-typed. scripts/collect-real.sh regenerates the fixtures from the same tools that built the machine (tsc, npm, pytest, node --test, this repo's public CI log, and the system dpkg log), and scripts/make-assets.py re-renders the README images from live CLI output. Where a tool is missing on your machine the scripts say so and keep the checked-in copy.

License

MIT