The ollama-handoff server lets an AI agent offload routine tasks to a local Ollama model, saving cloud costs and keeping frontier models focused on complex work. It can be configured via environment variables (Ollama URL, default model, context window, timeouts).
ask_local — Send a one-shot prompt to the local model for simple tasks (drafts, formatting, quick lookups)
chat_local — Conduct multi-turn conversations with the local model
summarize_local — Generate a structured summary (headline + bullets) of long texts, logs, or transcripts, with an optional focus hint
code_review_local — Perform a quick first-pass code review on a diff or code block to catch obvious bugs and style issues
draft_commit_message_local — Draft a conventional commit message from a git diff --staged output
extract_local — Extract structured data (e.g., URLs, function names, error codes) from unstructured text
list_models — List locally available Ollama models
server_info — View the server's effective configuration (default model, context size, Ollama URL) for debugging
Enables AI agents to offload routine tasks to local Ollama models, reducing cloud costs and frontier model context usage.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ollama-handoff summarize the errors in build.log"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that offloads cheap work from your cloud LLM agent to a local Ollama model.
Your frontier model (Claude, GPT, etc.) is brilliant and metered. A lot of the work it gets handed — summarizing a log, drafting a commit message, pulling every URL out of a file, a quick first-pass code review — doesn't need frontier reasoning at all. ollama-handoff exposes your local Ollama instance as a handful of purpose-built MCP tools, so your agent can route that work to a model on your own GPU — at zero cloud cost — and spend its (paid) reasoning budget on the things that actually need it.
This isn't a generic "wrap the Ollama API" server. Each tool ships with a baked-in system prompt and a description written for the calling agent, so the agent knows when to hand off and gets a tuned result back without re-stating instructions every call.
💸 Spend less. Routine offloads run locally and bill nothing.
⚡ Keep the big model focused. Summaries, extractions, and drafts don't eat its context or your budget.
🧠 Tuned, not raw. summarize_local, code_review_local, draft_commit_message_local, and extract_local come with reviewer/summarizer/extractor system prompts already dialed in.
🔌 Drop-in. One MCP registration; works with Claude Code, Claude Desktop, Cursor, and any MCP client.
🪶 Tiny & auditable. Two dependencies (mcp, httpx), fully typed, unit-tested, no telemetry.
Related MCP server: local-llm-delegation-mcp
Ollama running locally (ollama serve) with at least one model pulled, e.g. ollama pull qwen2.5-coder:14b.
Python 3.11+ (or just uvx, which manages it for you).
The fastest path is uv — no manual venv needed:
uvx ollama-handoff # run directly
# or
pip install ollama-handoff # then run: ollama-handoffclaude mcp add ollama-handoff -- uvx ollama-handoffmcp config block){
"mcpServers": {
"ollama-handoff": {
"command": "uvx",
"args": ["ollama-handoff"],
"env": {
"OLLAMA_DEFAULT_MODEL": "qwen2.5-coder:14b"
}
}
}
}A Dockerfile is included. The server speaks MCP over stdio, so run it
interactively (-i) and point it at your Ollama instance:
docker build -t ollama-handoff .
docker run --rm -i -e OLLAMA_URL=http://host.docker.internal:11434 ollama-handoffOn native Linux (no Docker Desktop), use --network=host with
OLLAMA_URL=http://localhost:11434.
Tool | What it does | When the agent should reach for it |
| One-shot prompt to the local model | Any handoff that doesn't need frontier reasoning |
| Multi-turn local chat | Handoffs needing more than one turn of context |
| Structured summary (headline + bullets) | Long files, logs, transcripts, docs |
| Quick first-pass review of a diff/code | Cheap pre-filter before a deep review |
| Conventional commit message from a diff | Routine commits |
| Pull structured items from unstructured text | URLs, function names, error codes, TODOs |
| List locally available Ollama models | Discovery / choosing a model |
| Report the effective configuration | Debugging setup |
All configuration is via environment variables set in your MCP registration:
Variable | Default | Description |
|
| Base URL of the Ollama server |
|
| Default model for handoffs |
|
| Context window in tokens |
|
| How long to keep the model resident in VRAM |
|
| Per-request timeout, seconds |
Once registered, you don't call the tools yourself — your agent does. A typical exchange:
You: Summarize the errors in
build.logand draft a commit for the staged fix.Agent: (calls
summarize_local(build.log, focus="errors and stack traces")anddraft_commit_message_local(git diff --staged)— both run on your GPU, nothing billed) → returns the summary + commit message.
git clone https://github.com/Michael-WhiteCapData/ollama-handoff
cd ollama-handoff
uv pip install -e ".[dev]"
ruff check .
pytest # tests use httpx.MockTransport — no running Ollama requiredSee CONTRIBUTING.md. Contributions welcome — especially new specialized handoff tools.
MIT © Michael Tierney
Send a one-shot prompt to a local Ollama model and return the response.
Use for any handoff where the cloud model's full reasoning isn't needed: drafts, boilerplate, simple extractions, formatting, quick lookups. Runs on the user's own GPU and consumes no cloud-LLM usage.
Args: prompt: The task / question. model: Override the default model. system: Optional system prompt to shape behavior.
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | Yes | ||
| model | No | ||
| system | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that it's a one-shot, runs locally on the user's GPU, and consumes no cloud usage. It doesn't mention potential errors or prerequisites like needing a running local Ollama instance, but it's adequately transparent for typical use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two short paragraphs and a bullet list), front-loaded with the purpose, and each sentence earns its place. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but indicated), the description does not need to detail return values. It covers use cases, parameters, and local execution. However, it omits prerequisites like having Ollama installed or the model available, which is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description includes an Args section that adds meaning: 'prompt: The task / question', 'model: Override the default model', 'system: Optional system prompt to shape behavior.' This significantly aids understanding beyond the schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends a one-shot prompt to a local Ollama model and returns the response. The verb and resource are specific, and it distinguishes from sibling tools like chat_local (multi-turn) and extract_local (specialized).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly lists use cases (drafts, boilerplate, simple extractions, formatting, quick lookups) and explains when to use it instead of cloud models: when full reasoning isn't needed, and it consumes no cloud-LLM usage. This provides clear guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Multi-turn chat against a local Ollama model.
Use when the handoff needs more than one turn of context. messages is
a list of {"role": "user"|"assistant"|"system", "content": str}.
| Name | Required | Description | Default |
|---|---|---|---|
| messages | Yes | ||
| model | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains the message format but omits key behavioral details such as state persistence, session management, or limitations like context window size.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, each serving a distinct purpose: purpose and usage/parameter format. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool involves multi-turn chat, critical details about conversation state (e.g., whether context persists across calls) are missing. An output schema exists but does not fully compensate for this lack of behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description adds value for the messages parameter by specifying the structure. However, the model parameter is not described, leaving a gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is for multi-turn chat against a local Ollama model, distinguishing it from single-turn or summarization siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use when the handoff needs more than one turn of context', providing clear guidance on when to use this tool over alternatives like ask_local.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Quick first-pass code review using the local coder model.
Catches obvious bugs, style issues, and risky patterns. Use as a cheap pre-filter before asking the cloud model for a deeper review.
Args: diff_or_code: A unified diff or a code block to review.
| Name | Required | Description | Default |
|---|---|---|---|
| diff_or_code | Yes | ||
| model | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description indicates it is a read-only, safe operation (quick review) but does not disclose any potential side effects, limitations, or failure modes. It adds moderate context beyond annotations (none) but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is three concise sentences: purpose, usage guideline, and parameter description. No wasted words, and key information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has two parameters (one required) and an output schema (not shown but exists). Description does not cover return values, but output schema handles that. It lacks mention of prerequisites or edge cases, but overall adequate for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains the 'diff_or_code' parameter as 'A unified diff or a code block to review,' adding clarity. However, the 'model' parameter is not described at all in the description, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it performs a 'quick first-pass code review' using the local model, catching 'obvious bugs, style issues, and risky patterns.' It distinguishes itself from a deeper cloud model review, and no sibling tool has a similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using this tool as a 'cheap pre-filter before asking the cloud model for a deeper review.' This gives clear context on when to use it. However, it does not explicitly mention when not to use it or name specific alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Draft a conventional-style commit message from a diff using the local model.
Cheap and fast — good for routine commits where the cloud model's analysis isn't needed.
Args:
diff: The output of git diff --staged or similar.
| Name | Required | Description | Default |
|---|---|---|---|
| diff | Yes | ||
| model | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It labels the tool as 'cheap and fast' but provides no specifics on side effects, authorization needs, rate limits, or output format. The behavior of the local model is not detailed, leaving significant gaps for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using a single sentence for purpose followed by a brief usage note and an Args section. It avoids redundancy, though the docstring format is slightly verbose for the minimal content. Overall, it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters and an output schema (not provided here), the description covers the core usage but omits details about return values, potential errors, or size limits. Given the lack of annotations, more behavioral context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the 'diff' parameter as 'The output of git diff --staged or similar', adding meaning beyond the schema. However, the 'model' parameter is not described, and the schema has no descriptions (0% coverage). The description partially compensates but leaves one parameter undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool drafts a conventional-style commit message from a diff using the local model. It specifies the verb 'draft', the resource 'commit message', the input 'diff', and the model type 'local'. This distinguishes it from sibling tools like summarize_local or extract_local.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use the tool ('good for routine commits') and when not ('where the cloud model's analysis isn't needed'). It implies an alternative (cloud model) but does not name a specific sibling tool. The guidance is clear and helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract specific information from a text block using the local model.
Good for pulling structured data out of unstructured text — function names, URLs, error messages, TODO comments, etc.
Args: text: The source text. what_to_extract: What to pull out, e.g. "all function definitions" or "every URL in the file".
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| what_to_extract | Yes | ||
| model | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It mentions using a 'local model' but does not disclose safety aspects (e.g., read-only nature), performance characteristics, or any side effects. The behavioral traits are minimal beyond the basic extraction operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two short paragraphs plus an Args list) and avoids unnecessary information. It is front-loaded with the main purpose. The Args section adds structure, though the model parameter is missing from that list.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but noted), the description does not need to detail return format. However, it is missing context about the model parameter and does not clarify how the extraction result is structured. Overall, it covers basics but leaves some gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 3 parameters with 0% description coverage. The description adds meaning for 'text' and 'what_to_extract' with brief explanations and examples. However, the 'model' parameter is not described, leaving its purpose unclear (though default null suggests it's optional).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies exactly what the tool does: extract specific information from a text block using a local model. It gives concrete examples like function names, URLs, error messages, and TODO comments, making the purpose clear. This distinguishes it from siblings like ask_local or summarize_local, which are not focused on extraction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states it is 'good for pulling structured data out of unstructured text', providing clear context for when to use. However, it does not specify when not to use it or mention alternative tools, such as using a different model or tool for broader summarization.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the Ollama models available locally.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Only states basic listing action. No annotations provided, and description fails to disclose behavioral aspects like read-only nature, performance, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous information. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple zero-parameter tool with output schema, the description is adequate. Could mention return type but schema covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; description doesn't need to add parameter info. Baseline 4 for zero parameters is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'List' and resource 'Ollama models available locally.' Distinct from sibling tools like ask_local, chat_local which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., server_info). Missing context about expected usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the server's effective configuration (model, context size, etc.).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description adequately states it returns configuration info without side effects. Could be more explicit about read-only nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single clear sentence, no wasted words. Front-loaded with verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complete for a zero-parameter info tool. Sufficiently describes return content; could add more details but adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, schema covers 100%. Description adds meaning by listing return examples (model, context size).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb 'Return' and resource 'server's effective configuration' with examples. Clearly distinguishes from sibling tools that perform other actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternatives, but implied as a simple info retrieval tool among active siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize a block of text using the local model.
Cheap offload for long files, logs, transcripts, or docs the cloud model doesn't need to fully ingest. Returns a concise structured summary.
Args: text: The content to summarize. Can be very long (context window is configurable). focus: Optional focus hint, e.g. "errors and stack traces" or "API surface only".
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| focus | No | ||
| model | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions handling long text and returning a structured summary, but does not disclose whether the tool has side effects, requires permissions, or other behavioral traits critical for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise two-paragraph structure with clear front-loaded purpose and bulleted parameters. No extraneous text, but the missing model parameter could be noted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists and no annotations, the description covers use cases and two of three parameters, but missing model documentation and behavioral disclosure leaves it incomplete overall.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% yet the description adds meaning for 'text' and 'focus' via the Args section. However, the 'model' parameter is undocumented, leaving a gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it summarizes text using a local model, positioning it as a cheap offload for long files. Use cases are listed, but it does not explicitly differentiate from sibling tools like ask_local or chat_local, which may also handle text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes when to use (long files, logs, etc.) and implies cost saving over cloud model, but does not provide explicit when-not or alternative recommendations among sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0ask_localchat_localcode_review_localdraft_commit_message_localextract_locallist_modelsserver_infosummarize_localScored across 8 tools
Each tool has a clearly distinct purpose: one-shot vs. multi-turn chat, code review, commit message drafting, extraction, summarization, model listing, and server info. No overlap.
Most tools follow the `<action>_local` pattern, but `list_models` and `server_info` deviate by lacking the `_local` suffix. All are snake_case and readable.
With 8 tools, the server covers a comprehensive set of common local offload tasks without overloading. The count is well-scoped for its purpose.
The tool set covers major offload scenarios: general queries, chat, code review, commit messages, extraction, and summarization. Minor gaps like model management or generic generation are absent but not critical for a handoff server.
Cost-optimized LLM model routing recommendations for autonomous AI agents
LLM Orchestration Agent
150+ vertical AI expert bots as agent tools. $1 bots run on YOUR machine - your data stays yours.
LLM Orchestration Agent 2
