llmtrim

What it does

You run Claude Code, Codex, Cursor, or your own app. Every turn, the tool sends a large request: system prompt, tools, history, raw command output. You pay for every token of that, including the parts that do not help the model.

A 200-line build log with two errors. Tool schemas resent on every call. JSON with hundreds of near-identical rows. That bulk is still billed.

llmtrim sits on your machine as a local proxy, trims the waste, and forwards a smaller request. The reply is unchanged. You keep the same tools and answers; you spend less.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)

Compression cannot raise your bill or break a request; worst case is zero savings. Everything runs locally, nothing is sent to us. In action →

For Claude Code the same install also wires /sub to route sessions through another subscription. Details →

Related MCP server: Headroom Mini

Get started

npm install -g @llmtrim/cli@latest && llmtrim setup
# open a new terminal, then keep working
llmtrim status

That's it. setup starts a local proxy, wires your shell, and enables recoverable tool-output shaping. When Claude Code is present, it also turns on /sub. You do not run a separate install for that.

Any tool that honors HTTPS_PROXY works (Claude Code, Codex, Cursor, Aider, your SDK). GitHub Copilot does not (certificate pinning). Full list →

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

# Package managers
brew install fkiene/tap/llmtrim
cargo binstall llmtrim
scoop install llmtrim
docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim

Full options: INSTALL.md.

Menu-bar / system-tray popover with the same savings numbers. Bundled in Homebrew, Scoop, and npm; setup can enable open-at-login. Open with llmtrim tray. On Linux desktops, interactive ensure can fetch the tray binary from the latest release (needs libwebkit2gtk-4.1 and libayatana-appindicator3).

Same technique as mitmproxy, scoped to LLM API hosts only. setup changes three things; llmtrim uninstall reverses all three:

1. Private CA in ~/.llmtrim/ (name-constrained; cannot intercept your bank or email)

2. Shell env: HTTPS_PROXY + CA trust

3. Login service: daemon at login

No API keys stored (your tool's auth is forwarded). Prompts never touch disk; only anonymous token counts. Recoverable tool results stay in bounded daemon RAM for five hours by default and disappear on restart. Full threat model: SECURITY.md.

llmtrim ca
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"

Day to day

llmtrim status     # savings + health  (aliases: monitor, gain)
llmtrim update     # new release, restart daemon, refresh integrations
llmtrim ensure     # match the recommended install state on this machine

After setup, update, or ensure, owned Claude Code /sub stays in sync with the binary.

Time series: llmtrim status --daily · --weekly · --monthly · --json · --csv.

In action

An agent ran a build. The tool returned 58 lines; two were errors. All 58 would have been billed.

4,662 chars → 978 (−79%). Errors stay verbatim. Repeated INFO lines fold into a template plus the values (lossless when the range is regular).

# before (noise + signal)
[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
… 28 more near-identical INFO lines …
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
… 25 more INFO lines …
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

# after (errors verbatim; INFO folded losslessly)
[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Try it on a request body of your own:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log folding is one stage. Others kick in on different waste:

Existing prompt-cache prefixes (cache_control) are left alone. On shell-capable agent turns, a newly arriving tool result may be shaped once before its first cache write; the exact raw result remains recoverable with the emitted llmtrim recall r_… command.

Stages run in savings order. Nothing under a cache_control marker is rewritten.

Default auto enables each stage only where it pays. safe is lossless-only. Config →

Claude Code

When ~/.claude exists, setup, update, and ensure wire /sub (and routed subagents). No separate install command for that.

Send Claude Code through CLIProxyAPI instead of Anthropic, or as a fallback hop when the current path fails. Login is CLIProxyAPI's TUI; decide for yourself whether that fits the provider ToS.

llmtrim sub on                  # install + start CLIProxyAPI, enable redirect
llmtrim sub on grok             # write opus/sonnet/haiku/fable → that CLI's models
llmtrim sub auth                # CLIProxyAPI TUI — sign in
llmtrim sub models              # official + live sidecar models
llmtrim sub map on opus grok-4.6
llmtrim sub chain anthropic,codex
llmtrim sub mode fallback       # try hops in order when the current one fails
llmtrim sub status
llmtrim sub off

Interactive: llmtrim status → tab 4 Sub — Off / Always / Fallback, then e to edit the input→output map (type to search CLIProxyAPI's catalog; s save, a add, d delete). [ ] rotate the fallback first hop. llmtrim update also updates CLIProxyAPI when you use it. Point LLMTRIM_CLIPROXY_URL at an existing instance to skip the managed sidecar.

Route only a delegated Claude Code subagent while leaving the parent window unchanged:

llmtrim agents install          # also installed/refreshed by setup, update, and ensure

Then ask naturally: Implement it using a Grok subagent, use Terra, or review this with GPT Terra. Provider-only agents preserve the child request's Claude tier through the configured mapping; an explicit model agent pins that provider model. Request-local agents override the window /sub and global policy only for their own requests. llmtrim agents uninstall removes only llmtrim-owned agent files and records an opt-out so ensure leaves them removed.

This window only (installed with ensure; includes subagents; survives /clear):

/sub on [optional:cli-or-model]  # gemini, codex, claude, grok, kimi, vertex, qwen, copilot, or a model id
/sub off
/sub status

Sidecar: ~/.llmtrim/cliproxy/ (binary + config). Auth: ~/.cli-proxy-api when present, else ~/.llmtrim/cliproxy/auth. Env: LLMTRIM_SUB, LLMTRIM_SUB_MODE, LLMTRIM_CLIPROXY_URL, LLMTRIM_CLIPROXY_KEY.

Anthropic /login vs claude.ai connectors: with global sub in always mode, by default llmtrim writes a dummy ANTHROPIC_AUTH_TOKEN into ~/.claude/settings.json (same idea as claude-code-proxy's ANTHROPIC_AUTH_TOKEN=unused) so Claude Code does not need a live Anthropic OAuth session. The MITM strips that dummy token and sends /v1/messages to CLIProxyAPI; non-messages Anthropic probes are answered locally so they never return 401 Invalid bearer token.

Claude Code treats any API-key auth as overriding claude.ai login, so claude.ai connectors are disabled while the dummy token is set. That is expected. To keep connectors (and accept Anthropic /login when the session expires):

llmtrim sub anthropic-login keep   # connectors OK; Anthropic login required
llmtrim sub anthropic-login skip   # default: no Anthropic /login; connectors off

Restart Claude Code after sub on / sub off / sub mode / sub anthropic-login for the settings change to take effect. Fallback whose first hop is "what's in use" still needs a real Anthropic login.

Use it as a CLI, MCP, or library

Same engine, no proxy required. No extra model calls; compress runs in-process.

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send --provider openai

use llmtrim_core::{compress, ir::ProviderKind};
let out = compress(request_json, Some(ProviderKind::OpenAi))?;

import llmtrim
out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")

import { compress } from "@llmtrim/js";
const out = compress(requestJson, "openai", "aggressive");

Bindings and WASM notes: crates/llmtrim-uniffi · crates/llmtrim-wasm.

llmtrim mcp install          # Claude Code
llmtrim mcp install --print  # paste into any client

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Tools: llmtrim_compress, llmtrim_compress_text, llmtrim_stats (same ledger as status).

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA:

No proxy: any MCP client can call llmtrim as tools (llmtrim mcp install), or use the CLI / library.

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Non-LLM connections pass through untouched.

Configuration

Default is fine for most traffic. auto inspects each request and picks compressors by shape (tools → agent, code → code, long Q&A → rag, else aggressive).

Override with LLMTRIM_PRESET=<name> or preset = "<name>" in $XDG_CONFIG_HOME/llmtrim/config.toml:

auto composes these per request shape, so most users never set them directly. Pick one when you know your traffic and want to skip shape detection:

Every stage is individually tunable via config flags; preset wins over individual flags. The full table is long; see the field list in config.rs or run llmtrim compress --help. The most useful knobs:

Env: LLMTRIM_PRESET (preset), LLMTRIM_CONFIG (config-file path).

These knobs are orthogonal to compression. Each resolves env-first, then from the config file, so set whichever fits. The env var wins when both are present.

extra_hosts entries must be exact hostnames (llm.acme.com, never a bare acme.com): each one widens the name-constrained MITM CA, which regenerates automatically on the next launch to cover them.

Claude Code options (subscription reroute) are under Claude Code.

export LLMTRIM_UPSTREAM_PROXY=http://host:port
# or with auth: http://user:pass@host:port  (redacted in logs)

Outbound calls use CONNECT + verifying TLS; the upstream only sees the encrypted stream. Looping to llmtrim's own listen address is rejected. Put the variable in the daemon's launch environment (launchd / systemd), not only your interactive shell. Profile secrets sit in plaintext.

Companion tools on another port (e.g. headroom) are fine.

The numbers

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

The same A/B on the standard academic suites, at a conservative shape-matched preset (qwen3-next-80b, paired 95% CI). Quality is the score on the original request vs the compressed one. GSM8K comes from the frontier above (n=12); the other three are the named benchmarks readers compare against (n=20 each):

Three rows compress with no quality loss; GSM8K is the one dip:

• BFCL drops the tool schemas the query doesn't need (a menu of 2 to 37 candidates per call).

• SQuAD v2 still answers its unanswerable questions correctly.

• TruthfulQA holds factual accuracy exactly: its ~75-token prompts are almost all answer text, so the safe preset finds nothing to cut.

• GSM8K trades −8pp of accuracy for −71% cost, so measure per workload before enabling its reasoning preset. ¹Its input goes negative because that preset injects a Chain-of-Draft instruction whose payoff is output-side (see the frontier table).

Evidence and a one-line reproduce (named-benchmark snapshot):

make -C crates/llmtrim-cli/bench data
(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \
   --corpus bench/data/squad2.jsonl --preset rag \
   --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Methodology, per-corpus frontier, and confidence intervals: crates/llmtrim-cli/bench/README.md. Reproduce it:

make -C crates/llmtrim-cli/bench data   # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)
(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite)  # live A/B across all corpora (needs OPENROUTER_API_KEY)
(cd crates/llmtrim-cli/bench/scripts && PYTHONPATH=. python3 -m benchkit.tools.chart)  # regenerate the chart + table

How it compares

Each tool compresses one slice of the request. llmtrim compresses input and output, leaves the cached prefix untouched to keep the prompt cache stable, and scores on whether the answer survives the cut, not on tokens removed. Both axes below use the o200k_base encoder and reproduce from this repo.

Input

Input reduction (deterministic) next to answer quality from a live A/B. Quality is the drop vs llmtrim at each tool's compared setting (✅ held, a statistical tie; ❌ significantly lower), so a big reduction with a ❌ means the tool bought tokens by losing answers:

Overhead is the median per-call compress time (Python wall-clock, not like-for-like CPU): Headroom and leanctx run ML on CPU here (faster on a GPU) and pay a one-time model load on top (~3 s and ~4 s); llmtrim is Rust and entroly is lexical, so neither does.

• auto is the quality-gated default; aggressive accepts lossy cuts where the gate holds.

• Headroom drops to 0% with its ML disabled (its routers no-op on prose).

• leanctx and entroly are lossy with no quality gate; entroly has no low-reduction mode.

Headroom ties at matched reduction (24-25%, n=30, not significant) but its longer answers hit the model's output-token limit and get truncated 12 times to llmtrim's 2, the output inflation behind its higher cost. leanctx (measured at 26%) and entroly (at 69%, its mildest) score significantly lower than llmtrim (n=20), and fall further at their headline reductions (vs-leanctx, vs-entroly).

Output

Output reduction by asking for terser responses, on a paid live call over 9 coding prompts:

The cost is the 949-token system prompt caveman resends on every request (right column); llmtrim's is 19 for nearly the same cut. Both still net-save here, so caveman's deeper cut comes out ahead only when the output it removes is worth more than the 949 tokens it adds back (vs-caveman artifact).

The tools stack: RTK shrinks CLI output, then llmtrim compresses the tool schemas on top. Full head-to-heads: crates/llmtrim-cli/bench/README.md.

Known limits

These are surfaced by the same A/B that proves the savings:

• Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.

• Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.

• The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.

• "Lossless" is input-side, not response restoration. A lossless stage preserves the information the model reads (a folded log run, a TOON-encoded array, an abbreviation legend the model decodes in-prompt), and the token gate reverts any input cut that doesn't pay off. The engine does not transform the model's response back to an original form.

Acknowledgments

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Retrieval & context: BM25 (Robertson & Zaragoza 2009, bm25); BM25+ (Lv & Zhai, CIKM 2011); RM3 (Lavrenko & Croft, SIGIR 2001); TextTiling (Hearst, CL 1997); TextRank (Mihalcea & Tarau, EMNLP 2004); MMR (Carbonell & Goldstein, SIGIR 1998); Submodular objective (Lin & Bilmes, ACL 2011); modified-greedy knapsack maximizer (Tang et al., SIGMETRICS 2021, arXiv:2008.05391); DPP diverse sampling (Chen et al., NeurIPS 2018); Lost in the Middle (arXiv:2307.03172); DSLR (arXiv:2407.03627).

Code: RepoCoder (arXiv:2303.12570); Hierarchical Context Pruning (arXiv:2406.18294); The Hidden Cost of Readability (arXiv:2508.13666); Minification token accounting (arXiv:2606.01326).

Tool output: Drain (He et al., ICWS 2017); Brain (Yu et al., IEEE TSC 2023); LogLSHD (arXiv:2504.02172).

Dedup & abbreviation: SimHash (Charikar, STOC 2002, gaoya); CompactPrompt (arXiv:2510.18043); Maximal repeats (arXiv:1304.0528) + Re-Pair (Larsson & Moffat, DCC 1999).

Output control: Chain-of-Draft (arXiv:2502.18600); TALE (arXiv:2412.18547).

Serialization: TOON (Token-Oriented Object Notation), Johann Schopplich.

Built on tiktoken-rs, tree-sitter, image, whatlang, hudsucker, rusqlite, and more.

Found a problem?

llmtrim doctor          # diagnose
llmtrim doctor --fix     # diagnose + apply repairs
llmtrim ensure          # same repair path

Each failing check names its fix. If a request was mangled, set LLMTRIM_CAPTURE_DIR and open an issue with the before/after pair.

If llmtrim saved you money, a ⭐ helps others find it.

Star History

Licensed under MPL-2.0. Use llmtrim freely in your stack, including commercially, with no source-disclosure obligation for your own code; the file-level copyleft applies only to modifications you make to llmtrim's own source files. Contributions via DCO sign-off.