Context Shrink

🌐 English · Türkçe

Context Shrink is a tiny MCP server that keeps requests to any OpenAI-compatible endpoint under the model's context limit. It measures the context before the call, then either drops the oldest messages or compacts them into a summary — instead of letting the server reject the request. It works with, among others:

• oMLX

• llama.cpp (llama-server)

• LM Studio

• vLLM

• the OpenAI API

• any OpenAI-compatible gateway

See Compatible backends for the full list.

Token counting: uses tiktoken when it is installed (cl100k_base by default) for a close approximation, and falls back to a tokenizer-free estimate (character count ÷ 4) otherwise. Either way it is a guard rail, not an exact budget — set CONTEXT_SHRINK_TOKENIZER (or the legacy OMLX_SHRINK_* / OPENAI_SHRINK_* names) to change the behaviour.

🔌 Compatible backends

The server only needs an OpenAI-compatible /v1/chat/completions endpoint — local model servers, self-hosted GPU engines and OpenAI gateways all qualify. Point it at the endpoint with CONTEXT_SHRINK_BASE_URL and CONTEXT_SHRINK_MODEL. The wire protocol, headers and error shapes are summarised in docs/reference/OPENAI-API-REFERENCE.md.

Quick config for the most common ones

Example with Ollama:

export CONTEXT_SHRINK_BASE_URL="http://127.0.0.1:11434/v1"
export CONTEXT_SHRINK_MODEL="qwen2.5-coder"

☁️ Remote / cloud OpenAI-compatible providers

Local is not the only option: every major provider exposes an OpenAI-compatible /v1/chat/completions endpoint, so point the server at it and set CONTEXT_SHRINK_API_KEY. All of these need a key or token.

• OpenAI

  • Base URL: https://api.openai.com/v1

  • Model: gpt-4o-mini

• Azure OpenAI

  • Base URL: https://{resource}.openai.azure.com/openai/deployments/{deployment}

  • Model: your deployment name

  • Uses the api-key header instead of Authorization — see below

• DeepSeek

  • Base URL: https://api.deepseek.com/v1

  • Model: deepseek-chat

• Groq

  • Base URL: https://api.groq.com/openai/v1

  • Model: llama-3.3-70b-versatile

  • Very fast

• Together AI

  • Base URL: https://api.together.xyz/v1

  • Model: meta-llama/Llama-3.3-70B-Instruct-Turbo

• Fireworks AI

  • Base URL: https://api.fireworks.ai/inference/v1

  • Model: accounts/fireworks/models/llama-v3p3-70b-instruct

• Mistral API

  • Base URL: https://api.mistral.ai/v1

  • Model: mistral-large-latest

• OpenRouter

  • Base URL: https://openrouter.ai/api/v1

  • Model: any model slug

  • Routes to 200+ models

• HuggingFace Inference Endpoints

  • Base URL: https://{your-endpoint}/v1

  • Model: your model

  • Private endpoint

• Cloudflare Workers AI

  • Base URL: https://api.cloudflare.com/client/v4/accounts/{account-id}/ai/v1

  • Model: @cf/meta/llama-3.3-70b-instruct-fp8

Azure OpenAI does not use a bearer Authorization header; pass its secret as a custom header:

export CONTEXT_SHRINK_BASE_URL="https://myresource.openai.azure.com/openai/deployments/my-deployment"
export CONTEXT_SHRINK_MODEL="my-deployment"
export CONTEXT_SHRINK_EXTRA_HEADERS='{"api-key": "azure-secret"}'

Additional static headers for any gateway are accepted through CONTEXT_SHRINK_EXTRA_HEADERS as a JSON object (for example an org id or a custom routing header).

Full catalogue

Anything below can serve as the summarisation endpoint; unlisted OpenAI-compatible endpoints work too.

• Local model servers:

  • Ollama

  • llama.cpp (llama-server)

  • LM Studio

  • Jan

  • GPT4All (Python API)

  • llamafile (--server)

  • KoboldCpp

  • Oobabooga Text Generation WebUI (OpenAI extension)

  • MLC LLM

  • Petals (distributed)

• GPU / production engines:

  • vLLM

  • Aphrodite Engine (vLLM fork)

  • Text Generation Inference (TGI)

  • SGLang

  • NVIDIA NIM / TensorRT-LLM

  • ExLlamaV2 + TabbyAPI

  • Xinference

  • OpenLLM (BentoML)

  • FastChat (LMSYS)

  • LocalAI

• Proxies & routers:

  • LiteLLM (100+ providers)

  • OpenRouter

  • Portkey Gateway

  • One API / New API

  • community Ollama-to-OpenAI proxies

• Libraries / custom stacks:

  • llama-cpp-python (llama_cpp.server)

  • Transformers + FastAPI

  • AnythingLLM / Msty

Related MCP server: PromptThrift MCP

📖 Overview

oMLX profiles for this class of models run 32K context (max_context_window: 32768). That is a deliberate choice: with a 4-bit KV cache it keeps the runtime footprint at roughly 17–20 GB per model and keeps prefill short on Apple Silicon. Raising the limit is not free — every extra token of context is paid for in memory and prefill time.

The cost appears in a long session. Three things keep growing until the total crosses the ceiling:

• the system prompt

• the conversation history

• the tool results

Once it crosses, the server refuses the whole request:

Prompt is too long: 39514 tokens > 32768 maximum context length

At that point the agent stops. Nothing was wrong with the question — the prompt was simply 6.7K tokens too big. The usual workarounds are all unsatisfying:

So this server takes the cheap path: keep 32K, send a smaller prompt.

Read the full walkthrough (why 32K, the three compaction modes, the environment configuration, and the known limitations):

English → · Türkçe →

✅ Recommendation

Use it if you run oMLX or another local OpenAI-compatible server with a fixed context window and you have hit Prompt is too long at least once. It is a single Python file and needs no GPU. estimate_context and trim_context are pure arithmetic — they cost nothing at inference time. compact_context only calls the local model when you ask it to summarise (or you can hand it a summary yourself and it calls nothing at all).

You probably do not need it if your context window is large enough for your sessions, or if your client already compacts history on its own.

🛠️ Tech Stack & Architecture

• Language & Runtime: Python 3.10+, standard library only

• Protocol: Model Context Protocol (MCP) over stdio transport

• Client: OpenCode (Code Mode groups tools under tools.Context_Shrink.*)

• Compaction Engine: Chunked summarisation via the local OpenAI-compatible endpoint

• Token Counting: optional tiktoken (default cl100k_base) with a character heuristic fallback — no hard dependency

• Dependencies: mcp (MCP SDK) required; tiktoken optional — no GPU

• Target Server: any OpenAI-compatible server — see Compatible backends

• Defaults: max_context_tokens = 32768, keep_last = 8, threshold = 0.8

✨ Features

• 🧮 Accurate-enough estimate — estimate_context returns the estimated token count, the remaining budget and a should_trim flag

• ✂️ Drop oldest messages — trim_context returns the message list with the oldest messages removed, keeping the most recent keep_last

• 🧠 Compact instead of drop — compact_context replaces the oldest messages with a summary, keeping leading system messages and the most recent keep_last verbatim

• 📝 Three compaction modes — summary provided, model-generated, or plan-only (needs_summary: true, no message content echoed back)

• 🌐 Optional real tokenizer — uses tiktoken when installed, otherwise a tokenizer-free estimate (chars ÷ 4); CONTEXT_SHRINK_TOKENIZER controls the choice

• 🔌 Plain MCP — any MCP client can use the server, not just OpenCode

• 🧩 Chunked summarisation — long histories are summarised in pieces so the local model does not overflow while summarising

• 🔒 No telemetry, no network calls except the optional local summariser request

📸 Screenshots

(No UI — this is a headless MCP server. Tool output is JSON on stdio.)

🏗️ Architecture

flowchart LR
    classDef client fill:#e3f2fd,stroke:#1976d2,color:#0d47a1
    classDef decision fill:#fff3e0,stroke:#ef6c00,color:#e65100
    classDef action fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
    classDef model fill:#fce4ec,stroke:#c2185b,color:#880e4f

    Client["🧑‍💻 Your coding agent"]:::client
    Client -->|"before every request"| Fits{"Does it fit the<br/>context window?"}:::decision

    Fits -->|"yes"| Model
    Fits -->|"no"| Choice{"What should happen?"}:::decision

    Choice -->|"measure only"| Measure["📊 Report the size<br/>nothing is changed"]:::action
    Choice -->|"drop oldest"| Trim["✂️ Shorten the history<br/>newest messages stay"]:::action
    Choice -->|"summarise"| Compact["🗜️ Summarise the oldest part<br/>recent messages stay word-for-word"]:::action

    Trim --> Model
    Compact --> Model
    Model["🤖 Local model<br/>(oMLX)"]:::model

compact_context in three modes

1. Summary provided — pass summary="..." and no model call is made. Use this when the agent (or the user) already wrote the summary.

2. Model-generated — pass model="..." (and optionally base_url), or set the environment variables below, and the server summarises the oldest messages for you. Long histories are summarised in chunks so the local model does not overflow while summarising.

3. Plan only — with neither, the call returns needs_summary: true, the exact summary_prompt to use, and the token counts. No message content is echoed back, so the result stays small; call again with summary=<text>. Pass model="" to force this step even when the environment sets a model.

compact_context returns the assembled messages list only in modes 1 and 2, and leaves the list untouched (compacted: false) when it is already within threshold unless you pass force: true.

No model endpoint required: modes 1 and 3 work without any CONTEXT_SHRINK_* variable. The environment variables only enable mode 2.

🚀 Installation & Quick Start

Install (one line)

The package is on PyPI: context-shrink-mcp.

pip install context-shrink-mcp     # the package + the `context-shrink` command

Or skip the install and let uvx fetch and run it — no checkout, no manual copy (install uv once). Register the server with OpenCode in one line:

# from PyPI
opencode mcp add context-shrink --global -- uvx --from context-shrink-mcp context-shrink

# or straight from GitHub (no package index needed)
opencode mcp add context-shrink --global -- uvx --from git+https://github.com/mahirfatih/context-shrink-mcp context-shrink

uvx resolves the mcp dependency in an isolated environment and runs the server over stdio. Pin a version with context-shrink-mcp@1.0.0 (PyPI) or …context-shrink-mcp@v1.0.0 (GitHub).

Enable model mode on the same command:

opencode mcp add context-shrink --global \
  --env CONTEXT_SHRINK_BASE_URL=http://127.0.0.1:8000/v1 \
  --env CONTEXT_SHRINK_MODEL=Qwen3.6-35B-A3B-4bit \
  --env CONTEXT_SHRINK_API_KEY=omlx-... \
  -- uvx --from context-shrink-mcp context-shrink

Without --global, opencode mcp add writes to the project config (./opencode.jsonc) instead of ~/.config/opencode/opencode.jsonc.

The numbered steps below are the repository-checkout path — use them to run the file directly or from a dedicated virtualenv.

1. Prerequisites

2. Install the SDK

pip install "mcp>=2.0"

3. Copy the server

Copy context_shrink.py into a directory of your choice (for example ~/.config/opencode/mcp/). A repository checkout also ships a four-line context-shrink.py shim that runs the same module, kept for existing setups that point at the hyphenated name.

Note: The python3 in the command array may not be the one you expect — depending on your setup it could be /opt/homebrew/bin/python3, /usr/bin/python3, or somewhere else. Verify the path with which python3 (or which python) and use the absolute path in the config instead of the bare python3 if needed.

4. Register with OpenCode

Add the server to your OpenCode config (~/.config/opencode/opencode.jsonc):

{
  "mcp": {
    "servers": {
      "Context Shrink": {
        "type": "local",
        "command": [
          "python3",
          "/absolute/path/to/context_shrink.py"
        ],
        "timeout": 180000
      }
    }
  }
}

Config changes apply without a restart — OpenCode watches config files and reloads MCP servers after a save. To force a full reload, run /reload.

5. (Optional) Enable model mode

Set the environment variables under environment in the MCP entry so compact_context can summarise with the local model. The values below are the ones this project was developed with — a local oMLX server on Apple Silicon, serving Qwen3.6-35B-A3B-4bit:

"environment": {
  "CONTEXT_SHRINK_MODEL": "Qwen3.6-35B-A3B-4bit",
  "CONTEXT_SHRINK_BASE_URL": "http://127.0.0.1:8000/v1",
  "CONTEXT_SHRINK_API_KEY": "omlx-...",
  "CONTEXT_SHRINK_TIMEOUT": "180",
  "CONTEXT_SHRINK_TOKENIZER": "auto"
}

A real setup (this project's development machine)

This server was written for, and is used daily on, an Apple Silicon Mac driving a local oMLX server — the 32K context profile it targets is documented in mahirfatih/omlx-m1-max-benchmark. The OpenCode entry used there runs the server from a dedicated virtualenv and points the summariser at the local model:

{
  "mcp": {
    "servers": {
      "Context Shrink": {
        "type": "local",
        "command": [
          "/absolute/path/to/context-shrink-venv/bin/python",
          "/absolute/path/to/context_shrink.py"
        ],
        "environment": {
          "CONTEXT_SHRINK_MODEL": "Qwen3.6-35B-A3B-4bit",
          "CONTEXT_SHRINK_BASE_URL": "http://127.0.0.1:8000/v1",
          "CONTEXT_SHRINK_API_KEY": "omlx-...",
          "CONTEXT_SHRINK_TOKENIZER": "auto"
        },
        "timeout": 180000
      }
    }
  }
}

The CONTEXT_SHRINK_API_KEY is the token the local oMLX server expects; it is redacted here. Replace the paths and the key with your own.

Configuration

compact_context reads its endpoint and model from the environment when the arguments are not given.

The canonical names are CONTEXT_SHRINK_*. The legacy OMLX_SHRINK_* names and the vendor-neutral OPENAI_SHRINK_* aliases are identical in meaning and are used only when every higher-precedence name is unset. Precedence:

CONTEXT_SHRINK_*  >  OMLX_SHRINK_* (legacy)  >  OPENAI_SHRINK_*

Common pitfalls

• An empty CONTEXT_SHRINK_MODEL puts compact_context into plan-only mode: it returns a needs_summary plan instead of calling a model.

• A too-small max_context_tokens makes every call look over-budget; set it to the model's real context window (the default is 32768).

• *_EXTRA_HEADERS must be a JSON object — a string, array or malformed JSON is ignored with a warning.

• A remote/cloud CONTEXT_SHRINK_BASE_URL forwards your messages to that endpoint; only point it at a server you trust.

• Out-of-range max_context_tokens values are clamped to 1…1000000 so a typo cannot exhaust memory.

6. Use it

• Call estimate_context with the current message list.

• If should_trim is true, call trim_context and send the returned list, or compact_context to keep the gist instead of dropping it.

• An agent can do this automatically before every request when the estimate crosses the limit.

Example estimate_context call:

{
  "messages": [
    { "role": "user", "content": "first message" },
    { "role": "assistant", "content": "first answer" },
    { "role": "user", "content": "second message" }
  ],
  "max_context_tokens": 32768
}

Example trim_context call:

{
  "messages": [ "..." ],
  "keep_last": 8,
  "max_context_tokens": 32768
}

Compaction flow, model mode:

{
  "messages": [ "..." ],
  "keep_last": 8,
  "model": "Qwen3.6-35B-A3B-4bit",
  "base_url": "http://127.0.0.1:8000/v1"
}

Compaction flow, agent mode (two calls):

1. compact_context({ "messages": [...], "force": true }) → needs_summary: true plus the summary_prompt.

2. The agent writes the summary, then calls again with { "messages": [...], "force": true, "summary": "<that text>" } → the compacted messages list.

Tool parameters

estimate_context also returns a tokenizer field naming the counter in use (for example tiktoken:cl100k_base or heuristic(chars/4)).

Minimal clients

examples/python-client.py and examples/js-client.js are runnable clients: each starts the server over stdio, lists its tools and calls estimate_context.

python examples/python-client.py                          # uses ./context_shrink.py
python examples/python-client.py /path/to/context_shrink.py

cd examples && npm install && node js-client.js           # same client in Node

Run the Python client with the same interpreter that has mcp installed; the Node client needs Node 18+ and npm install once.

📂 Project Structure

context-shrink-mcp/
├── context_shrink.py        # Single-file MCP server (the whole product)
├── context-shrink.py        # Four-line shim that runs the module above
├── pyproject.toml           # Packaging + the `context-shrink` console script
├── README.md                # English quick tour
├── README.tr.md             # Turkish quick tour
├── CONTRIBUTING.md          # Setup, tests, commit conventions
├── CHANGELOG.md             # Keep a Changelog release notes
├── docs/                    # Documentation tree (see docs/README.md)
│   ├── README.md            # Doc map
│   ├── guides/MIGRATION.md  # Rename / migration notes
│   ├── guides/RELEASING.md  # Version, build and publish steps
│   ├── reference/OPENAI-API-REFERENCE.md  # Endpoint, headers, errors
│   └── testing/TESTING.md   # Suites, runner, live tests
├── requirements.txt         # Runtime dependency (mcp)
├── requirements-dev.txt     # Dev dependencies (mcp, pytest, ruff, tiktoken)
├── pytest.ini               # pytest config (testpaths, python_files)
├── .ruff.toml               # Ruff lint configuration
├── .pre-commit-config.yaml  # Optional local git hooks
├── .github/workflows/ci.yml # CI: lint + compile + tests
├── test/                    # Test runner + suites
│   ├── test.sh              # Single test runner (ruff + pytest)
│   ├── README.md            # Test folder guide
│   ├── py/                  # Python suites
│   │   ├── context-shrink-tests.py       # Helpers, tokenizer, fixtures
│   │   ├── context-shrink-mcp-tests.py   # STDIO end-to-end round-trip
│   │   └── context-shrink-live-tests.py  # Live oMLX integration (opt-in)
│   └── js/                  # Node integration tests (MCP JS SDK)
├── examples/                # Runnable examples
│   ├── python-client.py     # Minimal MCP client: list tools + call one
│   └── js-client.js         # The same client in Node (MCP JS SDK)
├── LICENSE                  # MIT
└── .gitignore

🧪 Testing & CI

./test/test.sh              # install dev deps, then ruff + pytest
./test/test.sh --lint       # ruff only
./test/test.sh --no-install # skip the pip install step
./test/test.sh --only test/py/context-shrink-tests.py::test_compact_uses_provided_summary

• Runner: test/test.sh installs the dev dependencies, runs Ruff, then pytest. The interpreter is the project .venv when present, otherwise python3.

• Suites: test/py/context-shrink-tests.py covers the pure helpers (token estimation, trimming, compaction splitting and chunking), the tokenizer resolution logic and the env-var parsing — no network. test/py/context-shrink-mcp-tests.py adds an end-to-end round-trip over the real STDIO transport (handshake → tools/list → tools/call).

• Live tests (opt-in): test/py/context-shrink-live-tests.py and the Node suite in test/js/ hit a real endpoint. They are skipped unless CONTEXT_SHRINK_LIVE=1 is set and the endpoint answers, so the default run stays offline. The API key is read from CONTEXT_SHRINK_API_KEY, never stored in the repo.

• Details: docs/testing/TESTING.md has the full suite map, flags and live-test setup.

• Node tests: cd test/js && npm install && npm test runs the protocol tests through the official MCP JS SDK; the model test is gated by the same CONTEXT_SHRINK_LIVE flag.

• CI: .github/workflows/ci.yml runs Ruff, a py_compile check, and pytest. It is manual only — trigger it with workflow_dispatch (Actions tab → CI → Run workflow).

Run the server by hand to exercise it through a client:

python3 context_shrink.py   # starts the stdio server; Ctrl-C to stop

📚 Documentation

• README.md — this file.

• README.tr.md — Turkish translation.

• context_shrink.py — source of truth for tool schemas and behaviour.

• pyproject.toml — packaging and the context-shrink console script.

• pypi.org/project/context-shrink-mcp — the published package.

• CONTRIBUTING.md — development setup, tests, and commit conventions.

• CHANGELOG.md — release notes.

• docs/README.md — the documentation map.

• docs/reference/OPENAI-API-REFERENCE.md — the endpoint, headers and error shapes this server relies on.

• docs/testing/TESTING.md — the suites, the runner and the opt-in live tests.

• docs/guides/MIGRATION.md — renaming from oMLX Prompt Shrink.

• docs/guides/RELEASING.md — version, build and publish steps.

• examples/python-client.py — a runnable MCP client that lists the tools and calls one.

• examples/js-client.js — the same client in Node, built on the official MCP JS SDK.

• MCP Python SDK — the transport and MCPServer API this file is built on.

📝 Notes

• Default context limit: 32768 tokens — change it to match your profile.

• Trim policy: oldest messages first; the last keep_last messages are always kept.

• Compact policy: leading system messages are kept, the oldest of the rest are replaced by one system message holding the summary, and the last keep_last messages are kept verbatim.

• The tool surface is plain MCP, so any MCP client can use the server, not just OpenCode.

🔁 Migrating from oMLX Prompt Shrink

The project was renamed from oMLX Prompt Shrink to the vendor-neutral Context Shrink; the tool names are unchanged and the old repository URL redirects. The file path, tool namespace, repository and environment-variable mapping is in docs/guides/MIGRATION.md.

🔗 Background

The 32K profile and the Prompt is too long failure this server addresses are documented in mahirfatih/omlx-m1-max-benchmark: docs/reference/PROFILES.md explains why the context window is set to 32K, and the README has the recommendation section for this server.

🔐 Permissions & Privacy

Privacy: The server makes no network requests of its own except the optional summariser call to the local OpenAI-compatible endpoint you configure. No telemetry, no analytics, no data leaves the machine. All state is in-memory for the duration of a tool call.

Version: 1.0.0

📄 License

This project is licensed under the MIT License.