slimtoken-mcp exposes slimtoken's token-reduction pipeline as on-demand MCP tools — minify requests, count tokens, compress tool results, prune context, and inspect presets/config — without reimplementing any optimization logic.
slimtoken.optimize_messages — minify an Anthropic/OpenAI/Ollama request body (messages, system, tools), pair-safe and fence-aware, returning the reduced messages plus token counts and per-stage stats.
slimtoken.estimate_tokens — count tokens with the bundled real cl100k tokenizer, returning a total plus per-message breakdown.
slimtoken.prune_context — RAG-style pruning of a memory store: strip low-value text, retrieve warm entries by query, sliding-window summarize old turns, enforce a budget, and emit a ready-to-inject <cold_memory>/<recent_context> block.
slimtoken.minify_tool_result — lossy type-aware compression of a large tool result (dir listings, git output, logs, JSON, source) with a [slimtoken-compressed] header; only the content field changes.
slimtoken.inspect_budget — read-only budget check: per-section token counts, headroom, and whether the pair-safe pruner would drop leading messages.
slimtoken.get_config — return the active MinifyConfig built from SLIMTOKEN_* env knobs so you can see what slimtoken will do.
slimtoken.list_model_presets — list local-model presets by VRAM tier (4/8/16 GB) with usable context, optionally measure=true for live measured reduction.
slimtoken.high_context_presets — dense + MoE high-context configs per tier showing nominal context that fits in VRAM and the effective raw-token capacity after compression (best=true for the single largest).
Optimizes requests to Ollama's local API by reusing the OpenAI conversion to normalize to Anthropic's format, applying token minification, and converting back, reducing input tokens for local model inference.
Optimizes requests to the OpenAI API by converting bodies to Anthropic's canonical format, running the token-minification pipeline, and converting back, thereby reducing input tokens for OpenAI-compatible endpoints.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@slimtoken Can you optimize this request to use fewer tokens?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Every LLM call spends tokens — on a bloated system prompt, a tool schema you wrote twice, an old turn that no longer matters, "Sure!" at the head of every reply. slimtoken rewrites the request before it leaves your machine so the model sees less, charges less, and answers faster.
It's a small Python toolkit that runs three ways: as an always-on proxy in
front of any Anthropic / OpenAI / Ollama backend, as an MCP server any agent
can call, or imported as a plain library. Same code, same wins either way.
Since v0.5.0 it also ships slimtoken.memory — a persistent, disk-backed
memory + guard-rail layer for agents (formerly its own project, now merged in).
Alongside the request-body minify pipeline, slimtoken ships prompt reframe
— a 1-ms CPU pass that takes a rambling 200-word user prompt and returns a
tight 25-word instruction, with the original intent preserved by construction
(no LLM roundtrip). It's exposed as slimtoken.prompt_reframe, an MCP server
(slimtoken-reframe-mcp), and an Agent Skill (skills/prompt-reframe/).
A measured walkthrough: a one-table token-savings proof, the per-stage pipeline
at a glance, a worked before/after, install + first-call instructions, and links
to the deep reference. MIT-licensed; ships with orjson, xxhash, and
tiktoken so the token counts below are real, not guesstimates.
Every number below is printed by slimtoken's own real cl100k tokenizer from fixtures built into the package, so you can reproduce the whole table with one command, on your own machine, without trusting this file:
slimtoken modes --measurecode (default) and realtimeslimtoken runs in one of two modes, selected by SLIMTOKEN_MODE, and the mode
decides how much of a request is allowed to change. The command above prints what
each one saves.
code is the default, and it is the one for working on code. It removes a
tool result that is byte-identical to an earlier one, and prose in an assistant
turn older than the keep-last window. It does not touch tool results at all —
not an old one, not a new one. The file you just read and the file you read ten
turns ago both reach the model byte-for-byte, so nothing you fetched is ever
abbreviated behind your back. Tool schemas are left exactly as written, so how
the model fills in arguments is unchanged.
realtime is for talking to a model, not building with one. It elides old
user turns as well as assistant turns, cuts prose to 160 characters a turn, and
shortens even the newest tool result. That last property is why it must never run
agent work: an elided instruction still reads as a complete instruction, so the
model answers confidently from evidence it no longer has. On 2026-09-27 exactly
that — a skeletonised read the model was still waiting on — made an agent re-run
its own tool call.
Fixture | What it represents |
|
|
agent session | 8 turns, a different real file read each | 0.0% | −70.9% |
repeated reads | 8 turns, the same large file read every time | −73.1% | −85.3% |
spoken conversation | no tools, no code, no file reads | 0.0% | −59.5% |
The first and last rows say the same thing about the default mode: when nothing
is duplicated and no prose is long enough to elide, it saves nothing at all,
and that is the point. Eight distinct file reads are eight distinct pieces of
evidence and the default keeps every one of them. What it does still remove, and
removes losslessly, is a tool result you read twice — the middle row, where it
saves 73%. The realtime mode exists because it is willing to spend evidence for
speed, which is the right trade when you are talking and the wrong one when you
are working.
Neither mode is lossless in general, and "minimal" is a claim about which bytes move, so here is the whole of it:
|
| |
Your newest turns | byte-for-byte | shortened |
Fenced code blocks | byte-for-byte, always | byte-for-byte, always |
Tool schemas | untouched — tool-calling behaviour is unchanged |
|
Duplicate tool result | older copy stubbed; the same bytes are still later in the conversation, so nothing leaves it | same |
Old assistant prose | shortened, only beyond the last 4 turns | shortened to 160 chars, beyond the last 2 |
Old tool result | never touched — no result is ever rewritten | middle replaced with |
So code is lossless where it matters: every tool result, every fence and your
newest turns are untouched, and dedup loses nothing from the conversation as a
whole because the bytes it stubs are still there, later. The only thing it still
shortens is old assistant prose, and it marks every elision when it does — the
model is told bytes were dropped, and how many. realtime gives up the newest
turns to buy response speed, which is the trade you want when you are talking
rather than building.
The response-side filter strips lead-in filler ("Sure!", "Here is the code:",
"Let me know if you need anything else.", …) from the streamed head. On by
default — set SLIMTOKEN_FILLER=0 to disable. The token cap
(SLIMTOKEN_MAX_TOKENS) and stop sequences (SLIMTOKEN_STOP) are opt-in.
Scenario | Before | After | Reduction |
Short reply with filler lead-in | 44 tok | 38 tok | −13.6% |
Long reply with filler lead-in | 380 tok | 374 tok | −1.6% |
Clean reply (no filler) | 21 tok | 21 tok | 0% (nothing to strip) |
The two sides compose — a session that saves 68% on the way in and 1.6% on the way out saves both — but they are measured separately here on purpose, because the input side is the part that depends on which mode you chose.
The honest caveat: reduction is proportional to waste. A clean, short session with no repeated content and no filler gets ~0% — slimtoken never invents savings. The more a session re-reads files, repeats tool output, or carries verbose old turns, the more it saves. That's the point: it removes redundancy, not meaning.
Related MCP server: everything-slim
The proxy is the default. It sits in front of your model API and minifies every request automatically — the agent can't skip it, so you get the savings without relying on the model to remember to call anything.
# Local build — Cython-compiled by default (skips gracefully to pure Python)
git clone https://github.com/greyok00/slimtoken
cd slimtoken && ./scripts/install.sh
# or, from PyPI:
pip install slimtoken
# Default setup — the proxy (one command, reversible)
slimtoken install
slimtoken serve --upstream http://127.0.0.1:8080 # local llama-server
# or: slimtoken serve --upstream https://api.anthropic.com # cloud
# `slimtoken install` already wired ANTHROPIC_BASE_URL to the proxy.
# Point your client at it and every request is minified automatically.
# Nothing else to do.
# Alternatives (not the default — on-demand only):
# CLI: slimtoken optimize -i request.json
# MCP: slimtoken-mcpslimtoken install writes a marker block to your shell rc that exports
ANTHROPIC_BASE_URL to the proxy (prior value backed up to ~/.slimtoken/prev_env
and restored on uninstall). It never touches settings.json, CLAUDE.md, or
mcp.json, so removal is clean and fully reversible: slimtoken uninstall.
The proxy is the only surface that guarantees every request is minified. MCP tools and the CLI are on-demand — the agent has to choose to call them, and a busy agent will forget. The proxy rewrites the request at the API layer, so the savings happen whether or not the agent "remembers" slimtoken. If you want the token reduction to actually happen, route through the proxy.
Some setups don't want automatic minification (exact-fidelity debugging, or a model that must see raw tool output verbatim). Opt out cleanly:
Per-request passthrough: SLIMTOKEN_MINIFY=0 — raw passthrough, no rewrite.
Don't route through it: unset ANTHROPIC_BASE_URL (or point it at your
model directly) — the proxy only sees traffic that's sent to it.
Full removal: slimtoken uninstall — restores your prior
ANTHROPIC_BASE_URL and removes the marker block.
One proxy process runs one pipeline, and SLIMTOKEN_MODE picks which — so the
common case is one process with one env line, not two services:
You want | Set |
agent traffic: tool schemas and the system prompt arrive exactly as written |
|
a chat or voice endpoint, where raw context volume is the enemy |
|
When a box really does serve two very different backends at once — a cloud
endpoint where tool-call fidelity matters most, and a local model where volume is
the enemy — run two slimtoken serve processes on different ports, each pinned to
its own mode and upstream, each as its own user systemd unit, and point clients at
the matching port. Any SLIMTOKEN_* switch overrides the mode for that one knob,
so the stage table in "What it does" doubles as the per-instance config surface.
Every CLI command, one line each. slimtoken --help prints the same list.
Command | What it does | Key flags |
🚀 | Run the minify proxy (default upstream |
|
🔌 | Point |
|
🔧 | Remove the marker block, restore the prior |
|
🗜️ | Minify one request body, print before/after token counts |
|
📊 | Local-model presets by GPU VRAM tier |
|
🧭 | Print the |
|
📏 | High-context dense+MoE presets with effective context after compression |
|
⚙️ | Recommend llama-server args for a GPU + model (recommend-only) |
|
⏱️ | One request through a running proxy → |
|
🧩 | Lazy MCP stub server — advertises configured MCP servers as deferred tools, spawns the real server only when called |
|
Installed alongside the CLI ([project.scripts] entry points):
Entry point | What it does |
🛰️ | MCP stdio server exposing slimtoken's tools to any MCP host |
📝 | MCP stdio server for the prompt-reframe pipeline |
🚀 | Runs the proxy directly (same as |
A minify pipeline runs on each request. Which stages are on depends on the mode:
the default (code) keeps the stages that remove redundancy and turns off the
three that rewrite structure (tools, system, messages), so nothing the
model reasons about is reworded. The diagram shows the request lifecycle with the
t0–t4 latency boundaries the proxy records per request — proxy-side work
(ingress + optimize) is what slimtoken controls; model-side (forward → first
token → final token) is where real time goes.
sequenceDiagram
participant C as client
participant P as slimtoken proxy
participant B as backend / model
C->>P: POST /v1/messages (t0)
P->>P: read request (t0→t1)
P->>P: minify: the mode's stages · budget (t1→t2)
P->>B: forward minified body (t2)
B-->>P: first output token (t3)
P-->>C: stream raw bytes back (t3→t4)
Note over P: proxy-side = (t1-t0)+(t2-t1) ≈ 12 ms<br/>model-side = (t3-t2)+(t4-t3) — dominatesStage | What it does | On in | Lossy? |
🧰 tools | Drop | realtime | no |
📋 system | Collapse whitespace and duplicate banner lines outside code fences; preserve | realtime | no |
💬 messages | Collapse blank-line runs and trailing whitespace in text blocks; pass | realtime | no |
🔄 dedup | Collapse repeated | both | no* |
📝 distill | Shorten old assistant prose beyond the last | both | old assistant turns |
🎯 budget | Hard token cap ( | both | drops oldest |
🌐 dom (opt-in) |
| neither | yes |
🗜️ tool_compress | Type-specific reduction of large |
| yes, old results |
* dedup is lossless in practice — the latest copy is always kept verbatim; only stale duplicates are stubbed.
Safety guarantees — fenced code blocks (triple-backtick / ~~~) preserved
byte-identical; pruning is pair-safe (a tool_result is never orphaned from its
tool_use); the current turn's tool result is never compressed; identity-based
change detection returns unchanged content zero-copy; the grammar field is
stripped from request bodies.
The proxy above rewrites request bodies (tools, system, message history) on their way to a model. That's a different problem from tightening a single user prompt. If you're about to spend tokens on a rambling 200-word request that could be a sharp 25-word instruction, you'll want a rewriter first.
slimtoken.prompt_reframe is that rewriter. Pure CPU, no model roundtrip;
~1 ms per call. Five stages, called individually or as the bundled
frame_prompt pipeline:
flowchart LR
A[raw user prompt] --> B[classify_domain]
B --> C[reframe_prompt<br/>strip filler + dedupe]
C --> D[shrink_prompt<br/>TextRank-lite<br/>cap to word budget]
D --> E[minify_prompt<br/>cosmetic squeeze]
E --> F[build_system<br/>tight declarative system]
F --> G[tight prompt + system]Stage | What it does |
🏷️ | Keyword match into {business, professional, osint, cybersecurity, code, general}. Used to pick the right domain hint when composing the system prompt. |
🧽 | Strip 30+ conversational filler phrases ( |
✂️ | Rank sentences by relevance + length and pack the top-N until the word budget is met. Modes: |
🪶 | Collapse whitespace; drop redundant punctuation runs. Cosmetic only. |
🛠️ | Compose a tight system prompt from role + style + domain hints + up to 6 rules. Single short line so it doesn't waste context. |
Why TextRank-lite, not an LLM? It can't drop intent. The output is built from sentences that already appear in the user's prompt, ranked by their overlap with the prompt itself. A separate small LLM could paraphrase — but it costs a roundtrip and can quietly lose a detail. Use the reframe for intent-preserving shrink; pair it with an LLM only when you actually want a paraphrase.
Worked example:
Input |
|
Can you basically just tell me what is the answer really kind of like basically please help me with this. | Tell me what is the answer. |
1109 chars / 208 words / ~10 sentences about a Q3 revenue review | 212 chars / 27 words / 2 sentences with every actionable claim preserved |
from slimtoken.prompt_reframe import frame_prompt
tight, system, domain = frame_prompt(user_prompt, mode="balanced")
# → ("Revenue figure for Q3? Lock the plan or reforecast.",
# "Role: generalist. Style: terse. Domain (business): ...",
# "business")Three ways to use it:
# Python API — drop into any script, batch job, or web service
python -c "from slimtoken.prompt_reframe import frame_prompt; \
print(frame_prompt('rambling prompt here')[0])"
# CLI — pipe prompts in, get tight prompts as JSON out
python -m slimtoken.prompt_reframe "your rambling prompt here"
python -m slimtoken.prompt_reframe json "your prompt"
# MCP stdio — for any host that speaks MCP
slimtoken-reframe-mcp
# exposes: slimtoken.reframe.{classify_domain, reframe, shrink,
# minify, build_system, frame}When NOT to use it:
The prompt is already short (< 80 words) — the reframe is a no-op.
You want a semantic paraphrase the input doesn't already contain. Use an LLM for that.
You're a code agent and the prompt is mostly code — never touch code fences.
shrink_prompt has no hidden max_tokens default: mode decides the target
length unless an explicit int is passed.
→ Full algorithm in
skills/prompt-reframe/references/stages.md.
→ Agent Skill manifest: skills/prompt-reframe/SKILL.md.
slimtoken.memory)Compression makes each request cheaper; the memory layer makes the next
session smarter. slimtoken.memory is a disk-backed memory + guard-rail layer
for agent loops, in 22 modules under one import:
Two-tier memory — a hot NDJSON working set (append-only, unbounded) plus
cold JSON facts keyed by category, with atomic appends (crash-safe, a single
write() per line). Warm was retired as a storage tier in v0.4.2 — the
stats key stays zeroed and hot_to_warm_sync() survives only as a no-op
shim for old callers. A small SQLite DB (~/.config/cortexllm/cortexllm.db)
backs the coding-practices and extractor tables.
Agent-loop guard rails — retry + CircuitBreaker, LoopGuard
(detect runaway tool loops), PreFlightGate / verify_before_llm
(fail-fast checks before a model call), PostResponseVerifier (checks the
response after it lands), and ColdDistiller (folds cold-tier history down).
Response shaping — parse_response / collapse / stream_turn and
block models (TextBlock, ToolBlock, ArtifactBlock, …) for turning raw
model output into structured pieces.
MCP server — the same memory over MCP stdio for any agent host.
from slimtoken.memory import append, read_last, search, write_cold, cold_get
append("assistant", "fixed the retry backoff in the proxy") # → hot tier
search("retry backoff") # keyword search across hot memory
write_cold("decisions", {"fact": "keep q4_0 KV cache"}) # → cold tier
cold_get("decisions") # → the stored knowledge dictEverything is local files — no daemon required for the basics. The write path
is a single atomic os.write() per JSONL line, so a crash mid-session never
leaves a torn line behind.
MCP stdio server (needs the mcp extra):
pip install "slimtoken[mcp]"
python -m slimtoken.memory.mcp_serverConfig uses SLIMTOKEN_MEMORY_* env vars (retry limits, loop-guard
thresholds). Hot has no size cap — ALLOW_CAP is False, so hot grows
unbounded. Defaults are sane; nothing needs setting to start.
Where the data lives: ~/.config/cortexllm/ (memory tiers + SQLite DB) and
~/.cortexllm/ (runtime state). The directory name is historical — kept so
existing installs keep their data on upgrade. Everything is plain JSONL/SQLite;
copy the directory and you've backed up the memory.
Both halves of slimtoken have well-known competitors. This is the honest map — including where they win.
slimtoken proxy | LLMLingua / LLMLingua-2 | Provider prompt caching | Truncation / sliding window | ||
How it compresses | deterministic CPU rewrite of the request (minify · dedup · distill · budget) | a small LM scores and drops tokens or sentences | a fine-tuned model compresses context into gist vectors | caches the repeated prefix server-side | drops the oldest turns wholesale |
Model / GPU needed | none | yes — 1–3 B scorer, GPU recommended | yes — fine-tuned LLM | no | no |
Applied to every request | yes — sits in the request path | per-call; your code must invoke it | per-call | no — first call billed full | yes |
What's preserved | old user turns verbatim · tool pairs intact · fenced code byte-identical | intent (tokens reassembled, not verbatim) | gist, not text | nothing — same tokens, lower bill | nothing |
Response side | filler strip · token cap · stop sequences | none | none | none | none |
Adopt cost |
| pip + model download | research code | free (it's the provider's) | free |
Two honest notes:
Prompt caching is a complement, not a competitor. It cuts the bill on repeated prefixes, but the request still carries every token. slimtoken cuts the request itself — the two compose.
LLMLingua can compress harder in the general case — a model-driven rewrite squeezes more than a deterministic one — but it costs a model download and GPU time per call, and it reassembles your text. slimtoken's default path is deterministic and bounded in what it may touch: old user turns verbatim, tool pairs intact, every tool result byte-identical, fenced code byte-identical. Against plain truncation the difference is bigger: truncation throws old turns away; distill keeps a gist of each and preserves user turns verbatim.
slimtoken.memory | |||||
Storage | plain JSONL + SQLite, one directory | vector DB (SaaS-first) | Postgres + temporal graph | managed server | whatever you wire up |
Memories are made by | your agent appends explicit records | LLM extracts facts from chats | LLM builds a temporal knowledge graph | the agent edits its own memory | your code |
Search | keyword over hot; cold by category | semantic (embeddings) | graph + hybrid | semantic | per-implementation |
Model needed at rest | none | yes — extraction + embedding | yes | yes — it's an agent server | varies |
Runs fully offline | yes | OSS core, cloud-first | self-hosted Postgres | server process | library only |
Agent guard rails | retry · circuit breaker · loop guard · pre/post verify | none | none | none | none |
Backup | copy the directory | DB export | DB dump | DB dump | n/a |
The deliberate trade-off: no embeddings, no graph, no extraction LLM.
Semantic recall is genuinely better at "find related ideas" — if that's the
requirement, mem0 or Zep is the right tool. What slimtoken.memory is instead:
files you can read and copy, agent guard rails nobody else bundles, and writes
that survive a crash mid-line. Memory injected into a prompt is itself a
compression problem — which is why the tier system and ColdDistiller exist.
A repeated-reads session — the same large file read on every turn, which is the case the default mode still compresses:
$ slimtoken optimize -i request.json
tokens: 30480 -> 8206 (-73.1%)
stages: tools=0 system=False msgs=0 dedup=7 distill=0 budget_drop=0 tool_compressed=0What each stage did to that body:
Stage | Effect on the example |
🔄 dedup | 7 of 8 identical file reads → |
🗜️ tool_compress | did not run. It is off in |
📝 distill | nothing to do here — no assistant turn in this body exceeds the 4096-character limit |
The model still sees every file's content, in the latest result — just not the redundant copies. Measure your own payload:
slimtoken optimize -i request.json --json # full minified body, machine-readable
slimtoken presets --measure # recompute the reduction table on your machine
slimtoken latency # one request through a running proxy → t0-t4 printoutProxy latency is ~12 ms per request (optimize stage, warm) — negligible next to any LLM round-trip. The win is fewer tokens sent, not proxy speed.
Three levers on the streamed response. Filler-strip is on by default — it
removes lead-in filler with no downside. The token cap and stop sequences are
opt-in (they need explicit values). Set SLIMTOKEN_FILLER=0 to disable the
strip; with it off and no cap/stop set, the filter is inert (raw passthrough,
zero overhead).
Lever | Env / flag | Default | What it does |
🧹 filler strip |
| 1 | Drop lead-in filler ("Sure!", "Here is the code:", "Let me know if you need anything else.", …) from the response head. |
✂️ token cap |
| off | Truncate the stream at N output tokens, counted with the real tokenizer. |
🛑 stop sequences |
| off | Cut the stream at the first stop string (not emitted). |
slimtoken serve --upstream http://127.0.0.1:8080 \
--max-tokens 2048 --stop "END" --tool-compress
# or via env (filler needs no flag — it's on):
SLIMTOKEN_MAX_TOKENS=2048 SLIMTOKEN_STOP=END slimtoken serve --upstream http://127.0.0.1:8080The filler strip is a pending-buffer state machine — a phrase split across SSE chunks is still caught:
model emits: "Sure!\nHere is the code:\nprint(1)"
client sees: "print(1)"The token cap and stop truncation are applied to the streamed delta text, so a runaway completion is cut off at the source instead of flooding your context.
Set SLIMTOKEN_STATS_FILE=/path/to/stats.json and the proxy atomically persists
cumulative minify stats after every request (tmp + rename, so a crash never
corrupts the file):
{
"runs": 2,
"tokens_in": 3000,
"tokens_out": 700,
"tokens_saved": 2300,
"ratio_pct": 76.7,
"last_run_ts": "2026-08-11T09:30:00",
"last_saved_pct": 80.0,
"history_60s": [{"ts": "...", "saved_pct": 80.0}, {"ts": "...", "saved_pct": 73.1}]
}GET /metrics on the proxy returns cumulative token counts + the t0–t4 latency
buckets.
SLIMTOKEN_MODE picks one of two named starting points — the two tables under
Token reduction above show exactly what each one does to your bytes. Everything
else is a raw SLIMTOKEN_* env switch, and any switch you set explicitly overrides
the mode for that one knob: set SLIMTOKEN_KEEP_LAST=8 under code and the rest of
code still applies.
slimtoken modes prints both modes with their stage lists, and
slimtoken modes --measure prints what each one saves on the built-in fixtures. An
unrecognised mode name falls back to code and says so on stderr, so a typo can
never silently select the lossy mode.
The things you might actually want to do:
Turn it all off — SLIMTOKEN_MINIFY=0 (raw passthrough; for debugging or
when the model must see input verbatim).
Keep old user turns — code already does. Only realtime elides them, and
SLIMTOKEN_DISTILL_INCLUDE_USER=0 turns even that off.
Leave the lossy stage off — code does, by default: no tool result, old or
new, is ever rewritten. realtime turns it on, and SLIMTOKEN_TOOL_COMPRESS=0
turns it back off there too. It is the only stage that can drop bytes from a
file you read earlier, which is why the default mode does not use it.
Opt into HTML pruning — SLIMTOKEN_MINIFY_DOM=1, still off by default.
See the Config table for the full knob list. The single config
surface (build_config) is shared by the proxy, CLI, MCP server, and skill.
The proxy routes by URL path and the CLI/MCP accept a --format / format arg.
The minify pipeline is built around Anthropic's request shape; OpenAI and Ollama
bodies are normalized to that canonical form, minified, then converted back — a
thin adapter layer, no optimization logic is duplicated. The Anthropic path is
identity (zero work, byte-identical to before).
path | format | conversion |
|
| none (identity) |
|
|
|
|
| reuses the OpenAI conversion; Ollama-only fields ( |
Pair-safety is preserved across the round trip: an assistant tool call plus its
following role:"tool" replies become Anthropic tool_use + tool_result blocks,
the pipeline drops such pairs together, and the reverse conversion never orphans a
tool result from its call.
# proxy: point any of these at slimtoken; it detects the format from the path
export OPENAI_BASE_URL=http://127.0.0.1:8181/v1 # OpenAI clients → /v1/chat/completions
export OLLAMA_HOST=127.0.0.1:8181 # Ollama clients → /api/chat
slimtoken serve --upstream http://127.0.0.1:11434 # → your local Ollama
# CLI: minify an OpenAI/Ollama body directly
slimtoken optimize -f openai -i req.json
slimtoken optimize -f ollama -i req.jsonRecommended configs for common local models grouped by GPU VRAM tier, each with a
usable context (KV cache + overhead eat into the nominal max). The reduction
column is the live measured token drop the pipeline achieves on the
bloated payload — computed by the pipeline, not hand-waved
(slimtoken presets --measure).
VRAM | model | quant | usable ctx | reduction |
4 GB | Llama 3.2 3B | Q4_K_M | 8 192 | 73.1% |
4 GB | Qwen 2.5 3B | Q4_K_M | 32 768 | 73.1% |
4 GB | Phi-4 Mini | Q4_0 | 16 384 | 73.1% |
8 GB | LFM2.5-8B-A1B (MoE, 1.5B active) | Q4 | 32 768 | 73.1% |
8 GB | Qwen 2.5 7B | Q4_K_M | 32 768 | 73.1% |
8 GB | Gemma 3 12B | Q4 | 16 384 | 73.1% |
16 GB | Qwen 3 14B | Q4_K_M | 65 536 | 73.1% |
16 GB | Mistral Nemo 12B | Q4_K_M | 131 072 | 73.1% |
16 GB | Llama 3.1 8B | Q4_K_M | 131 072 | 73.1% |
Reduction is config-dependent, not model-dependent — the pipeline rewrites the request regardless of which model consumes it, so every tier shows the same number (the default mode on a bloated payload). On a short session with nothing repeated it is ~0% — and on a real agent session, where each turn reads a different file, it is ~68%. Tune the config with the
SLIMTOKEN_*env knobs, or pick a differentSLIMTOKEN_MODE, rather than by switching models.
Because slimtoken compresses input ~73%, a model's nominal context window holds
far more raw conversation than its size suggests. The effective capacity is
nominal_ctx / (1 − reduction). The presets below push each tier to the largest
nominal context that fits fully in VRAM (q4_0 KV, flash attention, full GPU
offload, --kv-unified) — computed by the backend optimizer, not asserted — and
show the effective raw-token capacity with compression. Each tier has both a
dense and a MoE/Mamba-hybrid option: hybrids (Qwen3.6-35B-A3B, LFM2.5-8B-A1B)
have ~5 KB/token KV vs ~30 KB/token for dense, so they reach far larger contexts on
the same VRAM.
slimtoken high-context # full table (all tiers, dense + MoE)
slimtoken high-context --vram-gb 16 # one tier
slimtoken high-context --vram-gb 16 --detail # + the llama-server commandstier | kind | model | quant | nominal ctx | total GB | margin | effective ctx |
4 GB | dense | Llama 3.2 3B | Q4_K_M | 16 384 | 3.67 | +0.33 | ~61 k |
4 GB | MoE | LFM2.5-8B-A1B | IQ2_S | 32 768 | 3.91 | +0.09 | ~122 k |
8 GB | MoE | LFM2.5-8B-A1B | Q4_K_M | 131 072 | 7.33 | +0.67 | ~487 k |
8 GB | dense | Llama 3.1 8B | Q4_K_M | 32 768 | 7.71 | +0.29 | ~122 k |
16 GB | MoE | Qwen3.6-35B-A3B | IQ3_S | 131 072 | 14.16 | +1.84 | ~487 k |
16 GB | dense | Llama 3.1 8B | Q4_K_M | 262 144 | 14.71 | +1.29 | ~975 k |
The 16 GB MoE row is capped at 128 k — the proven-stable value on a 16 GB card (256 k OOMs at ub=2048; 128 k@ub512 measured 13.7 GB). The 8 GB MoE row is capped at 128 k too (256 k is a razor fit, ~+0.04 GB margin — any VRAM spike spills it; 128 k leaves ~0.67 GB headroom). The 4 GB MoE at 2-bit is a quality trade-off — the dense 3B is usually the better 4 GB pick. All configs use q4_0 KV (
-ctk q4_0 -ctv q4_0) matching a proven local llama-server setup.
On-demand, not the default. The MCP server gives the agent tools it can call when it chooses. It does not minify every request — that's the proxy's job. Use the MCP server when you want the agent to minify on demand (or as a redundancy check that the proxy path is working). For guaranteed every-message minification, use the proxy (see Quick start).
slimtoken-mcp exposes the pipeline as MCP tools over stdio (the transport
every local MCP client uses). It is a thin adapter: every tool imports and calls
an existing core function — no optimization is reimplemented. The proxy and
the MCP server are independent processes that share the same library.
Tool | Calls | What it returns |
|
| minified messages/system/tools + token counts + per-stage stats |
|
| total + per-message token breakdown (cl100k, bundled) |
|
| a ready-to-inject |
|
| a type-compressed tool_result content block (lossy) |
|
| read-only token-budget headroom + would-drop count |
|
| the active MinifyConfig (built from |
|
| VRAM-tier presets, optionally with live measured reduction |
|
| high-context dense+MoE presets per tier, with effective context after compression |
optimize_messages, estimate_tokens, and inspect_budget accept a format
field (anthropic / openai / ollama); non-anthropic bodies are normalized to
canonical before the pipeline runs and returned in the caller's format.
Wire it into an MCP client's stdio config (example for Claude Desktop /
claude_desktop_config.json):
{
"mcpServers": {
"slimtoken": {
"command": "slimtoken-mcp"
}
}
}The server speaks the MCP JSON-RPC 2.0 protocol (initialize → tools/list →
tools/call), protocol version 2024-11-05, and is self-contained (stdlib only on
top of slimtoken's existing deps). It does no optimization itself — every call
dispatches to the core pipeline.
Client | MCP registration | Skill install |
Claude Code | Add to |
|
Codex |
|
|
OpenCode |
|
|
Verify with claude mcp list, codex mcp list, or opencode mcp list — the
server should show as connected. The skill directory is the same
skills/slimtoken-optimizer/ folder in every case; copy it into the client's
skill search path (see Agent Skill).
Proxy-first. The skill tells the agent that slimtoken runs as a proxy by default and every request is minified automatically — so the agent doesn't need to call anything. The CLI/MCP tools in the skill are the fallback for when the proxy isn't in the path (on-demand minification).
The skills/slimtoken-optimizer/ directory is a packaged Agent Skill (static
files a host agent runtime — Claude Code, ADK, Gemini CLI, Cursor — reads from
disk on activation, not a running process). It is model-agnostic and works
with local, cloud, or uncensored models: it rewrites the request, not the model.
skills/slimtoken-optimizer/
SKILL.md # L1 description (~50 tok) + L2 body (<800 tok)
references/optimization-policies.md # full stage list + pair-safety rules (loaded on demand)
scripts/optimize.py # wrapper: CLI primary, MCP stdio fallbackThe wrapper shells out to the slimtoken CLI when available, and falls back to a
one-shot MCP stdio call (slimtoken-mcp) when only the MCP server is installed —
so the skill works regardless of which surface the host has:
python3 skills/slimtoken-optimizer/scripts/optimize.py optimize -i req.json
python3 skills/slimtoken-optimizer/scripts/optimize.py presets --vram-gb 16 --measure
python3 skills/slimtoken-optimizer/scripts/optimize.py estimate -i req.jsonDrop the skills/slimtoken-optimizer/ directory into your agent's skill search
path and the host runtime surfaces it when a prompt matches "shrink / minimize /
trim tokens / context too long".
Defaults are the recommended values, and they depend on the mode — the column
below reads code / realtime. Set any switch to 0 to disable it. A switch
you set explicitly wins over the mode, so you can run code with one knob turned
up without restating the rest.
Env var | Default ( | Meaning |
|
|
|
| 1 / 1 | master switch; 0 = passthrough |
| 0 / 1 | minify tool schemas (strips |
| 0 / 1 | system-prompt whitespace + hedging-phrase removal |
| 0 / 1 | message whitespace collapse |
| 1 / 1 | stub byte-identical duplicate tool results |
| 1 / 1 | prose elision in turns older than the keep-last window |
| 131072 | 0 disables hard prune (distill still runs) |
| 4 / 2 | recent turns kept verbatim by distill and budget ( |
| 200 / 80 | only dedup tool results at least this long |
| 4096 / 160 | max chars per distilled old turn |
| 0 / 1 | 1 = also distill old user turns |
| (none) | comma-list of tool names to never minify |
| 0 / 1 | type-specific reduction of tool results older than the keep-last window (lossy); off in |
| 0 / 0 | lossy opt-in: prune large HTML tool_results |
| (unset) | output-token cap (enables output filter) |
| (unset) | comma-joined stop sequences (enables output filter) |
| 1 | strip lead-in filler ("Sure!", "Here is the code:") from the response head; 0 = off |
| (unset) | path to persist cumulative minify stats (runs, tokens in/out, saved %, 60-run history) as JSON |
| 0 | use HTTP/2 to the upstream |
| 8181 | listen port |
| (required to serve) | backend URL |
TLS for cloud HTTPS upstreams is handled by httpx (SNI; optional mTLS via
SLIMTOKEN_TLS_*; SLIMTOKEN_TLS_INSECURE=1 to skip verify). Lazy MCP — one
stub tool per configured MCP server in ~/.slimtoken/lazy_mcp.json, the real
server spawned on call — is available as a separate entrypoint.
flowchart TB
subgraph Fit[fit the model in VRAM]
W[weights<br/>-ngl 999 full offload] --> K[KV cache<br/>-ctk/-ctv q4_0]
K --> CB[compute buffer<br/>--kv-unified]
end
subgraph Speed[decode speed levers]
FA[flash attention<br/>-fa on] --> UB[big ubatch/batch<br/>-ub N -b N]
end
C[context window<br/>-c N] --> Fit
Speed --> Result[2-4× decode speedup<br/>~50-75% less wall-clock<br/>~2× context capacity]
Fit --> Resultslimtoken config-optimizer [--model /path/to.gguf] [--vram-gb 16] [--model-size-gb 12.7]
[--kv-per-token 5120] [--native-ctx 262144]config-optimizer inspects your GPU VRAM and model size, estimates weights
VRAM, KV cache, and the compute buffer, then recommends llama-server arguments
that fit without OOMing. It prints a ready-to-paste llama-server command plus
CORTEXAGENT_* env exports. It changes nothing itself — recommend-only.
Option | Flag | What it does | Potential gain |
🟢 Full GPU offload |
| All model layers on GPU. Decode is memory-bandwidth-bound — offloading even a few layers to CPU cripples speed. | The biggest decode lever; often several× vs partial offload. |
⚡ Flash attention |
| Fused attention kernel; lower VRAM, faster attention. | Largest on long context (up to ~2× on the attention portion). |
🗜️ KV cache quant |
| Halves KV cache size. | ~2× context capacity in the same VRAM; modest decode speedup. |
📏 Context window |
| Largest ctx that fits without OOM. | More usable history (capacity, not speed). |
📦 Ubatch / batch |
| Larger prompt-eval batch. | Faster input processing — compounds with slimtoken's input reduction. |
🔗 KV unified |
| Unified compute buffer (calibrated into the VRAM estimate). | Lower buffer overhead. |
🔀 Parallel slots |
| 1 slot = max per-request budget (raise for concurrency). | Higher throughput under concurrent load. |
Total potential: versus a naive baseline (partial CPU offload + fp16 KV +
no flash attention), enabling all of the above typically yields a 2–4× decode
speedup (≈50–75% less wall-clock per token) and ~2× context capacity. These
are typical llama.cpp ranges, not measurements taken by slimtoken — the real
figure depends on your starting config. config-optimizer computes the largest
safe values for your specific VRAM automatically.
⚠️ Estimate only — verify VRAM with
nvidia-smiunder a real prompt before trusting the margin. The compute buffer is calibrated for--kv-unifiedon a hybrid MoE; dense models or--kv-budgetchange the math.
Slimtoken is built on excellent open-source work — huge thanks to:
Project | Used for |
async HTTP client for the proxy and upstream calls | |
fast JSON (de)serialization on the hot path | |
python-xxhash (xxHash by Cyan4973) | fast content hashing for dedup |
real-tokenizer token counting (no whole-body estimates) | |
optional fast event loop | |
| |
the test gate (129 tests collected) | |
static type checking in the dev toolchain | |
the local inference stack |
Algorithm provenance — these aren't libraries, they're ideas slimtoken reimplements from scratch; credit where the ideas come from:
TextRank — the sentence ranker in prompt_reframe ("TextRank-lite") is
a from-scratch, graph-free variant of the algorithm from Mihalcea &
Tarau, "TextRank: Bringing Order into Text", EMNLP 2004
(paper). Only the ranking
idea is borrowed; no TextRank code is used.
xxHash — dedup hashing goes through python-xxhash (above), which binds Yann Collet's xxHash C library.
python3 -m pytest tests/ -q # 129 tests — everything, incl. memory + MCP + token-guard
python3 tests/test_all.py # 231 checks in 25 groups — the pipeline suite, per-check outputCover fence byte-identity, pair-safety, dedup, distill, ≥50% default reduction
on a bloated payload, real-tokenizer counting (no whole-body serialize),
single-pass equivalence, type-compressor pair-safety, output-filter truncation +
filler-strip, DOM pruning, stats persistence, async proxy end-to-end, /metrics
latency buckets, fast-path byte-identical passthrough, the full MCP stdio
handshake + every tool + the error paths — plus the memory layer: tier
lifecycle, atomic appends, env-var handling, retry/circuit-breaker, loop guard,
pre-flight/post-verify, and the memory MCP server.
MIT, Copyright (c) 2026 greyok00. See LICENSE.
Count tokens in a request body using the real cl100k_base tokenizer (bundled, offline). Returns total + per-message breakdown. The model arg is accepted for forward-compat but the count is cl100k-approximate for non-cl100k models.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | model name (informational only) | |
| tools | No | ||
| format | No | request format of the body (normalized to canonical before counting) | anthropic |
| system | No | ||
| messages | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavior: bundled/offline tokenizer, exact for cl100k, approximate for non-cl100k models. Without annotations, this carries the transparency burden; it also describes return shape (total + breakdown).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the main function. Each sentence adds value—tokenizer type, return output, model parameter caveat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the core operation, offline behavior, approximation caveat, and return structure. Lacks elaboration on the `tools`/`system` parameters, but the 'request body' phrasing implies they're included.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema only documents model and format (40% coverage), and the description adds meaning for `model` (informational, approximate for non-cl100k). But it does not clarify `messages`, `tools`, `system`, or `format` usage beyond the schema's minimal description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb-object structure: 'Count tokens in a request body'. The addition of 'real cl100k_base tokenizer' and 'total + per-message breakdown' gives specific scope and differentiates from sibling tools like optimize or prune.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for counting tokens, but doesn't explicitly state when to prefer this over sibling tools like inspect_budget or optimize_messages. No exclusions or alternative guidance provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the slimtoken config in use: the always-on MinifyConfig built from SLIMTOKEN_* env knobs (the single config surface shared by the proxy, CLI, and MCP server). Useful to see what slimtoken will do.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses the config is 'always-on' and 'shared by the proxy, CLI, and MCP server,' indicating a persistent, global read-only resource. The word 'Return' implies no side effects, but it does not explicitly state that or describe error behavior, which is a minor gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary action, and each clause adds value. It efficiently explains what the tool does, why it exists, and how it's configured without extraneous detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, the description provides enough context: it defines the returned object (MinifyConfig), its construction, and its shared role. This is complete for a simple config retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description adds context about the config's origin and scope, which is meaningful even though there are no parameters to document. It avoids redundancy with the empty schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Return the slimtoken config in use' with a specific verb and resource. It further distinguishes the tool from siblings by specifying the config is built from SLIMTOKEN_* env knobs and shared across proxy, CLI, and MCP server, which is distinct from the optimization/estimation/pruning tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage context: 'Useful to see what slimtoken will do.' This implies when an agent should call it, but it does not explicitly mention alternatives or when not to use it. Since the siblings are functionally distinct, the context is clear enough for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
High-context VRAM-tier configs (dense AND MoE) showing how slimtoken compression expands the effective context window. Each row gives the largest nominal context that fits fully in VRAM (computed by config_optimizer, q4_0 KV, flash attn, full offload) and the effective raw-token capacity = nominal_ctx / (1 - reduction). Use best=true for just the largest-effective-context preset of a tier.
| Name | Required | Description | Default |
|---|---|---|---|
| best | No | return only the largest-effective-context preset for the tier | |
| vram_gb | No | filter to one tier (4/8/16) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does a good job: it discloses the computation assumptions (config_optimizer, q4_0 KV, flash attn, full offload) and provides the formula for effective raw-token capacity. It also clarifies that rows represent presets and that best=true filters to the largest. Minor omissions include not stating default behavior when vram_gb is omitted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four sentences, front-loaded with the primary purpose, then details the computation, the formula, and a useful parameter tip. Every sentence adds value and there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (VRAM tiers, compression, formulas), the description provides sufficient context for an agent to understand what the tool returns and how the values are computed. The lack of an output schema is partially mitigated by the description's mention of 'rows' and the formula. The main gap is not specifying the full return set when vram_gb is omitted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no new parameter-specific syntax beyond the schema, but it does provide context for interpreting the output (formula for effective capacity). This is helpful but not essential, given the schema already explains both parameters clearly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as providing high-context VRAM-tier configs for both dense and MoE models, with a specific scope of showing how compression expands context. It distinguishes itself from siblings like list_model_presets by focusing on VRAM tiers and effective context capacity. However, it lacks an explicit verb like 'list' or 'get', which slightly reduces clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a specific usage hint for the 'best' parameter ('Use best=true...'), which is helpful. However, it does not explicitly state when to use this tool over alternatives like list_model_presets or get_config. The context is implied by the title and description, but no exclusions or alternative comparisons are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read-only token-budget inspection: counts system/tools/messages, reports headroom against a token_budget, and whether the pair-safe pruner would drop any leading messages. Does not modify the body.
| Name | Required | Description | Default |
|---|---|---|---|
| tools | No | ||
| format | No | request format of the body (normalized to canonical before inspection) | anthropic |
| system | No | ||
| messages | Yes | ||
| keep_last | No | ||
| token_budget | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description shoulders the full burden of behavioral disclosure. It explicitly states the tool is read-only, does not modify the body, and details what it counts and reports. This is solid transparency for an inspection tool, though it omits specifics like return format or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with 'Read-only token-budget inspection', and every phrase adds value. It avoids redundancy and clearly communicates the tool's core behavior without fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and output schema, the description covers the essential operational semantics: what it inspects, what it reports, and its side-effect-free nature. It does not elaborate on input format normalization or parameter details, but for a read-only inspection tool, the provided context is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has only 17% description coverage, so the description must compensate. It meaningfully explains the role of 'messages', 'system', 'tools', and 'token_budget' through the counting and headroom reporting, but it does not clarify 'format' (beyond schema enum) or 'keep_last', leaving gaps for those parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb ('inspect') and resource ('token-budget'), and enumerates the exact outputs (counts, headroom, pruner decision). It distinguishes itself from sibling tools by emphasizing its read-only inspection nature, contrasting with the more action-oriented siblings like optimize_messages and prune_context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context through 'Read-only' and 'Does not modify the body', signaling that this is for inspection, not modification. However, it does not explicitly name alternative tools or provide when-to-use vs. when-not-to-use guidance, so it stops short of the highest score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List recommended local-model presets by GPU VRAM tier (4/8/16GB), each with a usable context. With measure=true, enriches each row with the live measured token reduction on a bloated payload (run by the pipeline itself).
| Name | Required | Description | Default |
|---|---|---|---|
| measure | No | run the pipeline to measure real reduction | |
| vram_gb | No | filter to one tier (4/8/16) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that 'List' is a read-only action and explains that measure=true triggers the pipeline to measure live token reduction, implying additional processing. It does not mention auth or rate limits, but for a listing tool this is reasonable context. The description adds valuable behavioral detail beyond the bare schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two sentences that are tightly packed with relevant information. The main purpose is front-loaded, and the second sentence adds a conditional behavior. There is no fluff or redundancy; every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with two optional parameters and no output schema, the description provides sufficient context: what the tool lists, the filtering criterion, and the optional enrichment behavior. It mentions that each preset 'has a usable context', giving a hint of the return content. It could be slightly more explicit about the exact response structure, but it does not leave a major gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds meaning to the 'measure' parameter by explaining it 'enriches each row with the live measured token reduction on a bloated payload (run by the pipeline itself)', which is not in the schema. It also confirms the VRAM tiers (4/8/16GB) that align with the vram_gb parameter, providing useful context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'List recommended local-model presets by GPU VRAM tier (4/8/16GB)', which is a specific verb+resource+scope statement. It clearly distinguishes this tool from sibling tools like optimize_messages, prune_context, and minify_tool_result, which are mutation/optimization tools, whereas this is a read-only listing operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use this tool: when the user needs recommended local-model presets filtered by GPU VRAM tier. It also explains the optional measure=true behavior. However, it does not explicitly contrast with the sibling 'high_context_presets' tool or provide exclusions, so it stops short of full alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compress a large tool_result content block using type detection (directory listing, git output, logs, JSON, source). LOSSY — emits a compact representation plus a [slimtoken-compressed] metadata header. Pair-safe by construction (only the content field changes).
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | tool_result content (string or list of text blocks) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden and transparently discloses that the operation is LOSSY, emits a metadata header, and is pair-safe by construction (only the content field changes). This goes beyond a simple action verb and informs the agent about side effects and limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action, and every clause earns its place. It conveys purpose, behavioral traits, and safety in a compact form without repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, no output schema, and no annotations, the description covers the core required context: it states what is compressed, how (type detection), the lossy nature, the output metadata marker, and the pair-safe guarantee. It does not specify exact output structure but sufficiently sets expectations for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter is already well-documented. The description adds minimal extra meaning beyond noting 'large' content and type detection, which does not change the parameter syntax or semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb 'Compress' and resource 'tool_result content block', and lists concrete type detection categories (directory listing, git output, logs, JSON, source). This clearly differentiates it from sibling tools like optimize_messages or estimate_tokens.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for large tool_result content blocks but does not explicitly state when to use this tool over alternatives or when not to use it. There is no reference to sibling tools or exclusions, so it stays at the 'implied usage' level.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reduce prompt size while preserving message structure and tool-call validity (pair-safe, fence-aware). Returns the minified messages plus token counts. Lossy by default (distill + tool-result compression); disable stages via the SLIMTOKEN_MINIFY_* env knobs.
| Name | Required | Description | Default |
|---|---|---|---|
| tools | No | tool definitions to minify | |
| format | No | request format (anthropic=identity; openai/ollama are normalized to canonical, minified, then returned) | anthropic |
| system | No | system prompt (string or list of text blocks) | |
| messages | Yes | Anthropic-style messages array | |
| max_input_tokens | No | override the token_budget (hard prune cap) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It explicitly states 'Lossy by default (distill + tool-result compression)' and mentions how to disable stages via SLIMTOKEN_MINIFY_* env knobs, which is meaningful for a mutation-like tool. It could go further by explaining 'distill' or authentication needs, but the disclosure is strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loaded with the core purpose in the first sentence and key behavioral caveats in the second. Every word earns its place, with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description mentions the return value (minified messages plus token counts) and the lossy default with env-knob control, which is sufficient for a tool with a well-documented schema. However, it does not elaborate on what 'pair-safe, fence-aware' entails or how it differs from sibling tools, leaving minor gaps for deeper contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter already has a descriptive schema entry, including format behavior and default. The description adds no additional parameter-level detail beyond what the schema provides, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb+resource: 'Reduce prompt size while preserving message structure and tool-call validity (pair-safe, fence-aware).' This clearly states the operation and its distinctive scope, distinguishing it from sibling tools like prune_context by emphasizing lossy compression and structural preservation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use the tool (for reducing prompt size while preserving structure) and gives config context via env knobs. However, it does not explicitly mention when not to use it or name alternatives among siblings, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
RAG-style context pruning for a memory/conversation store: strip low-value text, retrieve warm entries relevant to a query, sliding-window summarize old turns, and enforce a token budget. Returns a ready-to-inject / prompt block.
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | current query for relevance retrieval | |
| cold_data | No | cold memory keyed by category (each value is a list of entries) | |
| max_tokens | No | ||
| warm_entries | Yes | warm/conversation entries (role+content dicts) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full transparency burden. It outlines the internal algorithm (strip, retrieve, summarize, enforce) and the return type, but it does not disclose whether the operation is read-only, whether it modifies the underlying store, or any side effects such as data loss. This ambiguity prevents a higher score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose, and includes only essential details. Every phrase contributes to understanding the tool's behavior and output, with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the return value (a prompt block with cold_memory/recent_context), which is essential since there is no output schema. It also covers all key operations and the overall context of use. However, it does not specify the exact structure of cold_data beyond the schema, nor does it mention the required parameter warm_entries, though the schema handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes 3 of 4 parameters (75% coverage), and the description adds marginal context by linking 'query' to relevance retrieval and 'warm entries' to sliding-window summarization. However, it does not add significant new meaning or format details beyond what the schema provides, especially for cold_data and max_tokens.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'prunes context' in a RAG-style manner, detailing specific operations: stripping low-value text, retrieving warm entries, summarizing old turns, and enforcing a token budget. This distinguishes it from sibling tools like optimize_messages or estimate_tokens, which focus on different aspects of token management.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for context pruning in memory/conversation stores but does not explicitly state when to use this over alternatives. No comparisons are made to sibling tools like optimize_messages or minify_tool_result, so the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.3.3slimtoken.estimate_tokensslimtoken.get_configslimtoken.high_context_presetsslimtoken.inspect_budgetslimtoken.list_model_presetsslimtoken.minify_tool_resultslimtoken.optimize_messagesslimtoken.prune_contextScored across 8 tools
Several tools overlap in purpose: optimize_messages, prune_context, and minify_tool_result all reduce token counts, while list_model_presets and high_context_presets both list presets. However, descriptions clarify distinct use cases (full prompt vs. memory store vs. tool result; standard vs. high-context configs), so most boundaries are navigable.
Most tools follow a clear verb_noun pattern (optimize_messages, estimate_tokens, prune_context, minify_tool_result, inspect_budget, get_config, list_model_presets), but high_context_presets breaks the pattern as an adjective_noun phrase, causing a minor inconsistency.
With 8 tools, the set is well-scoped for a specialized token optimization server. Each tool addresses a distinct aspect (estimation, minification, pruning, budget inspection, config, presets) without unnecessary bloat or redundancy.
The tool surface covers the core lifecycle of token optimization: estimating, minifying, pruning, inspecting budgets, and retrieving config/presets. Minor gaps exist (e.g., no explicit tool to reverse or restore compressed content), but these are not essential given the lossy and config-driven design.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
