mcp-llm-offload

An MCP server that offloads light LLM work from Claude (or any MCP client) to a model you control — a local LLM (LM Studio, Ollama, llama.cpp) or any OpenAI-compatible provider (OpenRouter, xAI Grok, OpenAI, Groq, Together…). Save frontier-model quota on the cheap, non-critical stuff.

Why

Frontier models are great, but a lot of day-to-day agent work is light: summarize this log, classify this ticket, pull fields out of this blob, rephrase this sentence. Paying frontier-model rates (and quota) for that is wasteful.

mcp-llm-offload exposes a handful of MCP tools that forward those tasks to a backend of your choosing. Because LM Studio, Ollama, llama.cpp, OpenRouter, Grok, OpenAI, Groq and Together all speak the same /v1/chat/completions API, one tiny server talks to all of them — and you can switch backends with an env var or override per call.

Related MCP server: Just Prompt

Features

• 🔀 Provider-agnostic — one server, any OpenAI-compatible endpoint. Presets for the common ones; bring-your-own for the rest.

• 🏠 Local-first — defaults to a local LM Studio; no API key required for local backends.

• 🎯 Purpose-built tools — ask, summarize, classify, extract, health — each shaped for a light task, not just a raw chat passthrough.

• 🧭 Per-call routing — every tool takes optional provider and model args, so the cheap stuff goes local and the slightly harder stuff can go to Grok/OpenRouter without reconfiguring.

• 🩺 Actionable errors — connection, timeout, auth, 404-model, and rate-limit failures come back as plain, fix-this-next strings instead of stack traces.

• 📦 Single file, zero install — PEP 723 inline deps mean uv run llm_offload_mcp.py just works.

• 🤖 Claude Code subagent included — an optional llm-offloader agent that auto-routes light work for you.

Supported providers

Use any name you like for a custom provider: set FOO_BASE_URL (and FOO_API_KEY if needed), then call a tool with provider="foo".

How it works

Claude Code ──stdio──▶ mcp-llm-offload ──HTTP /v1/chat/completions──▶ your backend
   (frontier)            (this server)                                 (local / Grok / OpenRouter …)

The server is a thin, well-behaved MCP front-end. It resolves which backend and model to use (per call → env → preset), folds any system instruction into the user turn for maximum template compatibility, calls the endpoint, and returns clean text (or an Error: … string).

Quick start

1. Prerequisites

• uv (recommended) — or Python 3.10+ with pip.

• A backend: a running local server (e.g. LM Studio → Developer ▸ Start Server) or an API key for a hosted provider.

2. Get it

git clone https://github.com/jonpol01/mcp-llm-offload.git
cd mcp-llm-offload

Run it standalone to confirm it starts (it serves MCP over stdio, so it will wait for a client — Ctrl-C to exit):

uv run llm_offload_mcp.py

No uv? pip install mcp httpx then python llm_offload_mcp.py.

3. Register with Claude Code

The MCP server name you choose here becomes the tool prefix (mcp__<name>__ask, …). The bundled subagent expects the name offload, so use that unless you also edit the agent.

Local LM Studio (point it at a LAN host if LM Studio runs on another machine):

claude mcp add offload \
  -e LLM_PROVIDER=lmstudio \
  -e LMSTUDIO_BASE_URL=http://localhost:1234/v1 \
  -e LLM_MODEL=your-local-model-id \
  -- uv run /absolute/path/to/llm_offload_mcp.py

OpenRouter:

claude mcp add offload \
  -e LLM_PROVIDER=openrouter \
  -e OPENROUTER_API_KEY=sk-or-... \
  -e LLM_MODEL=meta-llama/llama-3.3-70b-instruct \
  -- uv run /absolute/path/to/llm_offload_mcp.py

xAI Grok:

claude mcp add offload \
  -e LLM_PROVIDER=grok \
  -e XAI_API_KEY=xai-... \
  -e LLM_MODEL=grok-2-latest \
  -- uv run /absolute/path/to/llm_offload_mcp.py

Or, equivalently, in a JSON MCP config (.mcp.json, Claude Desktop, etc.):

{
  "mcpServers": {
    "offload": {
      "command": "uv",
      "args": ["run", "/absolute/path/to/llm_offload_mcp.py"],
      "env": {
        "LLM_PROVIDER": "lmstudio",
        "LMSTUDIO_BASE_URL": "http://localhost:1234/v1",
        "LLM_MODEL": "your-local-model-id"
      }
    }
  }
}

4. Verify

In Claude Code, run the health tool (or ask Claude to). You should see the resolved provider, base URL, and the list of models the backend reports.

Tools

Every generation tool accepts provider and model to override the configured default for that single call.

Configuration

All configuration is via environment variables — none are required if the defaults (a local LM Studio) suit you and you pass model per call.

See .env.example for a copy-paste starting point.

The Claude Code subagent (optional)

agents/llm-offloader.md is a ready-made subagent that proactively routes light work to this server and hands anything heavy or correctness-critical back to the main agent. It runs on a cheap dispatch model (haiku) so the routing costs almost nothing and the work lands on your backend.

Install it by copying into your agents directory:

# user-wide
cp agents/llm-offloader.md ~/.claude/agents/
# or per-project
mkdir -p .claude/agents && cp agents/llm-offloader.md .claude/agents/

Its tools: list references mcp__offload__*, so it requires the server to be registered under the name offload.

Examples

Ask Claude things like:

• "Summarize this 4k-word changelog into 5 bullets — offload it."

• "Classify each of these 30 support messages as bug / feature / question using the local model."

• "Extract name, date, and total from this receipt text as JSON via the offloader."

• "Use the offloader on OpenRouter to rephrase this paragraph." (per-call provider="openrouter")

Troubleshooting

Development

uvx ruff check .          # lint
uv run --with mcp --with httpx python -c \
  "import importlib.util as u; s=u.spec_from_file_location('m','llm_offload_mcp.py'); m=u.module_from_spec(s); s.loader.exec_module(m); print('ok', m.mcp.name)"

CI (GitHub Actions) runs the same lint + import smoke test on every push and PR.

Contributing

Issues and PRs welcome. Keep the server single-file and provider-neutral; new providers are usually just one row in the PROVIDERS registry.

License

MIT © John Paul Soliva