Skip to main content
Glama

whittle-mcp

Trim what your agent reads, and measure what it kept.

Coding agents spend most of their tokens re-reading tool output: test runs, logs and stack traces, big JSON, diffs and API specs. Every result stays in the conversation and is re-sent on every later turn. whittle-mcp shrinks that output without losing the facts, and it proves what it kept.

  • Five trimmers: test output, logs, JSON, git diffs, OpenAPI documents.

  • Nothing is lost. What it removes is replaced by a short marker such as [... 312 lines omitted: whittle expand a1b2c3d4 lines=45-356 ...], and expand returns exactly that part on request.

  • Two ways to use it: an MCP server (uvx whittle-mcp) and a pipe for any shell (pytest 2>&1 | whittle tests).

  • Measured, not claimed: every trimmer is scored on how many tokens it saves and how many must-keep facts survive, on a public corpus. See docs/BENCHMARK.md.

Try it in your browser, nothing to install or send anywhere: https://huggingface.co/spaces/builditwithgk/whittle-mcp (paste a test log, JSON or diff and see what is kept).

Install and use

pip install whittle-mcp          # or: uvx whittle-mcp
pip install "whittle-mcp[tokens]"   # optional: exact token counts with tiktoken (otherwise an estimate, labelled)
pip install "whittle-mcp[yaml]"     # optional: read YAML OpenAPI documents

As a filter in any agent's shell (the simplest way, and it works with every agent):

pytest 2>&1 | whittle tests          # failures and the summary, not 4,000 dots and passes
npm test 2>&1 | whittle tests
git diff | whittle diff              # lockfiles and generated files collapse to one line each
curl -s https://api.example.com/items | whittle json
whittle log build.log                # repeats counted, long tracebacks shortened, every distinct error kept
whittle openapi spec.json --index    # one line per operation
whittle openapi spec.json --path '/pets*' --method get      # Git Bash on Windows rewrites a leading /: use MSYS_NO_PATHCONV=1 or --operation-id / --tag
whittle expand a1b2c3d4 lines=45-356       # get back what a marker omitted: paste the marker as printed (or use --lines 45-356)
whittle report                       # tokens saved so far, net of anything you expanded

The receipt ([whittle tests] 12,431 -> 912 tokens (-92.7%) | full output: whittle expand a1b2c3d4 | counter: tiktoken:o200k_base) goes to stderr, so pipes stay clean. Note that in a pipe the exit code is whittle's, not the test runner's: use set -o pipefail (bash) if a script needs the runner's status.

As an MCP server:

claude mcp add whittle -- uvx whittle-mcp --tools tests,log      # Claude Code, with just two trimmers
{ "mcpServers": { "whittle": { "command": "uvx", "args": ["whittle-mcp"] } } }

The tools take a file path (or, if you start the server with --allow-run, a command). A text argument exists for small inputs, but sending a big text through a tool call costs the tokens you meant to save, so do not use it for large output. Refusals (a path outside the root, commands switched off) and command failures (not found, timed out) come back with their reason. A large input that is not in the format you asked for is not echoed back: you get its first lines and a hint. Typical use: the agent runs pytest > out.log and calls trim_tests(path="out.log").

Tool

What it does

trim_tests

pytest, unittest, jest, mocha, RSpec, TAP, go test, cargo test: failures with their assertions and locations, plus the summary. Any other format is returned unchanged

trim_log

adjacent repeats collapsed and counted, long tracebacks keep their last frames, the first three errors of each shape stay (and the rest are summarised as "N more like ...")

mask_json

object keys stay; arrays show their first three items plus up to three later ones that add a new key or a new kind of value; long strings are shortened; the result is still valid JSON

trim_diff

lockfiles, generated or vendored files and whitespace-only changes collapse; long runs of unchanged lines are cut

slice_openapi

an index of operations, or only the operations you ask for plus the schemas they use

expand

returns what a marker omitted, by line range or JSON path

report

tokens saved this session, net of anything re-expanded, and money saved at your prices

Related MCP server: latent-gate

What it costs, honestly

The tool definitions themselves are sent every turn. Measured with mcp-reportcard: all seven tools cost about 1,570 tokens per turn; --tools tests,log (plus expand) costs about 610. Expose only the trimmers you will use. On the benchmark corpus an average call saved several thousand tokens, so it pays for itself when the agent trims a long output every few turns; if your agent rarely produces long output, it will not.

Results (public corpus, methodology and caveats)

Token counter: tiktoken o200k_base. "Naive" is head-and-tail truncation to the same token budget as whittle.

Corpus

Samples

Tokens saved (whittle)

Facts kept: whittle

Facts kept: naive truncation

Development corpus (tests, logs, JSON, diffs, OpenAPI)

48

53.3%

100%

84.4%

Held-out set (written after the trimmers)

16

61.3%

100%

85.8%

Savings depend on the input and are token-weighted, so large outputs dominate: logs shrank about 96%, OpenAPI slices 84%, diffs 60%, test output 47%. Short outputs save little: half the diffs and most short pytest -q runs in the corpus saved under 15%, and on the held-out set's short test runs the saving was 7%. Honest limits of these numbers: the corpus is small and was built by the author; the "must-keep facts" were chosen by the author (failing test ids and assertion lines, error lines, changed files, JSON key paths, operation ids); the development corpus was in view while the trimmers were written, which is why the held-out set exists; and end-to-end agent success with and without trimming has not been measured yet. Treat the numbers as a regression suite and a starting point. Samples that break it are the most useful contribution you can make.

Cost, in dollars, with your prices

Provider prices change, so the tool never bakes one in. whittle report and the benchmark use an editable price table (whittle prices prints the format) and a stated model: a token you keep out of the context is saved on the turn it would have arrived and on every later turn it would have been re-sent, at the cache-read rate if you use prompt caching. The default table holds clearly labelled example numbers, not any provider's real prices.

Limits

  • It uses rules, not a model: common formats are handled well, and a format it does not recognise (a test runner it does not know, text that is not JSON) is returned unchanged, never mangled.

  • Errors that differ only in numbers (order ids, codes) count as the same shape after the third occurrence; the rest are summarised (N more like ...) and are one expand away. Only adjacent repeats are collapsed and counted inline, and a failure with no recognisable keyword (no ERROR, no exception name, none of the common phrases such as No space left on device or exited with code 137) can fall into an omitted range.

  • Terminal colour codes are removed, and a line longer than 1,000 characters keeps its first 700 and last 300.

  • trim_diff output is for reading, not for applying. Lockfile bodies are collapsed, so a newly added dependency is not visible until you expand it: expand lockfiles when you review dependency changes. Files under dist/, build/ or vendor/ are treated as generated even if you wrote them. An indentation-only change is collapsed only for languages where it cannot change meaning; in Python and YAML it is shown.

  • JSON: numbers and duplicate keys are kept exactly as written; arrays show the first three items (plus any that reveal other keys) and a string marker such as ... +497 more items, and long strings are cut at 120 characters. Input nested more than about a thousand levels is returned unchanged.

  • Windows: files written by PowerShell 5.1's > (UTF-16) and UTF-8 files with a BOM are read correctly, and output is always UTF-8. Text you pipe from PowerShell 5.1 is re-encoded by PowerShell itself (its $OutputEncoding is ASCII); set $OutputEncoding = [Text.Encoding]::UTF8 or pass a file name instead.

  • Installing pulls in the MCP SDK and its dependencies, even if you only use the command line.

  • Small inputs are returned unchanged: there is nothing to gain from a short log (under about 40 lines) or a short test run (under 12 lines). CRLF line endings and carriage-return progress output are turned into plain lines first.

  • Token counts differ between providers; tiktoken is used when installed, otherwise an estimate (chars/4, labelled).

  • An agent only benefits if it calls the tool. The shell filter and a one-line instruction in your agent settings (below) help.

  • The CLI keeps the full text you trim in one shared temp folder ($WHITTLE_DIR, default whittle-store in your temp directory) so expand works for 24 hours; the MCP server keeps it in memory unless you pass --store-dir, so ids from one do not work in the other. whittle report adds up everything in that folder, including earlier projects. Logs can contain secrets: use --no-store to keep nothing, or point WHITTLE_DIR at a folder per project.

  • The MCP tools read files only inside --root (default: the current folder) and run commands only with --allow-run.

How other tools relate

Command-output compressors such as RTK, sqz, szr and ContextZip, and MCP servers such as TokenDiet MCP, solve a neighbouring problem; Claude Code and the Anthropic API also clear old tool results. They report large savings (their own figures). whittle-mcp has not been benchmarked against them yet; whittle bench --tool NAME=COMMAND runs any stdin-to-stdout trimmer through the same corpus and fact check, and a fair, reproducible comparison is a good first contribution.

Tell your agent to use it

Add to CLAUDE.md, AGENTS.md or your agent's rules:

For long command output, pipe it through whittle first: `pytest 2>&1 | whittle tests`, `git diff | whittle diff`,
`whittle log build.log`. Markers like "[... N lines omitted: whittle expand ID ...]" can be expanded with `whittle expand ID`.

Benchmark and contribute

pip install -e ".[dev]"
pytest -q
whittle bench                              # the development corpus (needs a git clone: the corpus is not in the pip package)
whittle bench --corpus bench/holdout       # the held-out set
python bench/make_corpus.py --source-repo <a Python repo with tests and git history>   # rebuild the corpus

Add a sample by putting name.txt and name.json (kind, options, must-keep facts) in bench/corpus/<kind>/. See CONTRIBUTING.md.

License

MIT. Built by Gopi K Aitham at Scaleup Solutions.

Available Tools

7 tools
expandA
Read-onlyIdempotent

Get back text that a trim omitted. Give the id and, ideally, the line range or JSON path from the marker.

Inputs: id (required), path, lines. Returns: result

mask_jsonA
Read-only

Show the shape of a large JSON document: every key path stays, long arrays and strings are shortened. Use it to see what a big API response contains; fetch details with expand and a path such as $.items[3].

Inputs: path, text, command, max_depth, +2 more. Returns: result

reportA
Read-onlyIdempotent

Tokens saved in this session, net of anything re-expanded, and the money saved at EXAMPLE prices.

Inputs: cached, later_turns. Returns: result

slice_openapiA
Read-only

Read a large OpenAPI document cheaply: with no filters you get an index of operations; with filters you get only the matching operations and the schemas they use.

Inputs: path, tags, text, index, +4 more. Returns: result

trim_diffA
Read-only

Shrink a git unified diff: lockfiles, generated files and whitespace-only changes collapse to one line each, and context around changes is cut. The result is for reading, not for applying.

Inputs: path, text, command, context. Returns: result

trim_logA
Read-only

Shrink a log or stack trace: repeated lines collapse, long tracebacks keep their last frames, every distinct error line stays. Use it instead of reading a whole log.

Inputs: head, path, tail, text, +1 more. Returns: result

trim_testsA
Read-only

Shrink test-runner output (pytest, unittest, jest, go test, cargo test) to the failures and the summary. Use it instead of reading a whole test log. Omitted parts can be fetched with expand.

Inputs: path, text, command, max_failure_lines. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 7 tools

    Disambiguation5/5

    Each tool has a distinct target artifact or meta-operation: logs, test output, diffs, JSON, OpenAPI, expansion, and reporting. The closest overlap is trim_log vs trim_tests, but their input formats and outputs are clearly differentiated. expand and report are also clearly separate as retrieval and analytics tools.

    Naming Consistency4/5

    All names use snake_case and are readable. The trim_* family is consistent, while mask_json and slice_openapi follow a verb_noun pattern, but expand and report are bare verbs. This is a minor deviation rather than a confusing mix.

    Tool Count5/5

    Seven tools is well-scoped for a token-reduction server. Each tool covers a distinct artifact type or meta-operation, and none feels redundant or padded.

    Completeness4/5

    The surface covers common large-output cases: tests, logs, diffs, JSON, and OpenAPI, plus expansion and savings reporting. A generic trim tool for arbitrary text and a way to list active markers/sessions would round it out, but the main workflows are covered.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      B
      maintenance
      Shrinks logs, JSON, docs and web pages before your AI reads them: offline, deterministic, no API key. Folds repetitive log lines (12,037 to 127 tokens on a 400-line log) while code, URLs and quoted strings come back byte-for-byte. Also fetches URLs as clean text, counts tokens exactly, and has optional local Ollama tools for image and text compression.
      59 PyPI
      25
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables MCP-compatible developer environments to shrink verbose tool outputs—test logs, git diffs, directory listings, and file dumps—before they reach the context window, preserving compiler errors, failure signatures, and decisive anchors. It runs either as direct callable tools or as a transparent proxy wrapping other MCP servers such as filesystem, git, or terminal providers.
      1
      MIT
    • A
      license
      Not graded
      quality
      Not graded
      maintenance
      Enables agents to deterministically shrink multi-turn conversation history and prompts by stripping verbose boilerplate and filler tokens while preserving critical tool parameters and named entities. This lets users cut inference costs by roughly 40-60% without losing essential context, running either as a native MCP server over JSON-RPC stdio or as an importable zero-dependency Python module.
      MIT