Provides web search capabilities through DuckDuckGo (with optional Tavily key) as a built-in tool for the agent loop.
Supports locally running Ollama models for tool-calling tasks through the OpenAI-compatible API.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@openlocal read notes.md and summarize the project structure"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A tool belt that small local models can actually use — and the receipts to prove it.
pip install openlocal
openlocal quickstartThat's it. quickstart finds the model server you already run — ollama, LM Studio,
llama.cpp, vLLM — writes your config, measures which tool-calling protocol your model can
actually drive, and runs a real task so you see it work:
$ openlocal quickstart
openlocal quickstart - looking for a model server...
found ollama at http://127.0.0.1:11434/v1 with 6 model(s)
wrote ~/.openlocal/models.toml
default model: qwen3-4b (served as qwen3:4b)
measuring which tool protocol this model can drive...
-> native tool calling
demo: Read notes.md and tell me how many lines it has.
tool file_read
final
The file notes.md has 3 lines.
You are set up. Try:
openlocal run "list the files here and summarise what this project is"
openlocal eval # score this model on 15 deterministic tool tasks
openlocal mcp # serve these tools to Claude Code over MCP🪶 Zero dependencies | Pure standard library. Installs in seconds, runs on a Raspberry Pi, and can't be broken by somebody else's release. |
🔌 Works with what you have | Anything that speaks the OpenAI |
📊 Measured, not claimed | 10 models from 9 vendors, scored on 17 deterministic tasks. No LLM judge — every check is a file diff or a regex. |
🧰 One registry, two surfaces | The same 8 tools power an MCP server (Claude Code, any MCP client) and a local-model agent loop. |
🩹 Tuned on real failures | Every ergonomic rule here was paid for by a model breaking. The lessons table is the part you can't get anywhere else. |
🧩 Add a vendor by editing TOML | New model family? Edit |
Related MCP server: Hermes MCP Server
Same tasks, same tools, one model at a time, two runs each. tools is the protocol each
model was measured driving — not the one its docs claim.
model | vendor | tools | passed | pass rate | bad json | tok/s |
| Alibaba | native | 17/17 | 100% | 0 | 40.4 |
| z.ai | native | 17/17 | 100% | 0 | 39.9 |
| Alibaba | native | 17/17 | 100% | 0 | 16.6 |
| NVIDIA | native | 16/17 | 94% | 0 | 54.9 |
| native | 16/17 | 94% | 0 | 50.3 | |
| OpenAI | native | 16/17 | 94% | 2¹ | 45.3 |
| IBM | native | 15/17 | 88% | 4 | 66.0 |
| Meta | native | 14/17 | 85% | 2 | 33.4 |
| Mistral | native | 14/17 | 82% | 0 | 11.1 |
| Microsoft | native | 13/17 | 79% | 0 | 61.5 |
¹ gpt-oss's harmony parser rejects some of its own tool calls (llama.cpp answers 500).
The client's retry ladder recovers every one — the retries column in
RESULTS.md keeps that honest.
Reproduce it: openlocal leaderboard --repeat 2
A 3B model doesn't fail because it's stupid. It fails because tool output is a prompt, and most tool output is written for humans. Everything below was measured — kept when the score went up, reverted when it didn't.
what broke | the fix | measured effect |
"What is line 300?" → model answered line 311 | number every line: | fixed in one turn, across models |
Model walked a 500-line file one line per turn | footers state facts, never commands ("file continues" ≠ "call again with…") | phi: 12 wasted steps → 2 |
Prose instead of a tool call | prose with no tool = the final answer (native only) | phi: 51 malformed replies → 5 |
…but "Sure, I'll use file_grep" is a plan | intent detection: plans get nudged, answers get accepted | runs stopped ending on intentions |
| prose call-syntax is parsed and executed | wasted turns eliminated |
| JSON repair: brace scan, tail closing, Python literals, fragment merge | phi bad_json 5 → 0 |
Model wrote the file before reading the source | stale-write detection at finish time | llama: 13 → 15 / 15 |
Model guessed "44" and repeated it when nudged | one turn of | granite recovered the task |
Read a 4-line CSV perfectly, said the sum was 40 | a | granite: +2 tasks |
Wrote the output file without ever reading the source | thin-air-write detection at finish time | qwen-4b over MCP recovered |
Copied the |
| edit_code: 10/10 models pass |
Same call, forever (A→B→A→B) | repeat guard replays the cached result | llama pagination loop gone |
Server 500s on its own tool-call syntax | retry ladder: as-asked → warmer → same turn without | gpt-oss: 11/15 → 14/15 |
Reverted: one more system-prompt rule | — | granite 12/15 → 11/15. Fewer rules win. |
openlocal eval --via-mcp reruns the whole eval suite with the tools served over a real
stdio MCP server — schemas crossing the wire, results as content blocks, a subprocess per
task. The scores match the in-process numbers, which is the point: what Claude Code
experiences is what was measured.
pip install "openlocal[mcp]"
claude mcp add openlocal -- openlocal mcpThe same 10 tools your local model uses, now in Claude Code. finish stays behind — it's
loop control, not a capability.
openlocal quickstart # find a server, configure, probe, demo
openlocal run "goal" # agent loop over your tools
openlocal run "goal" --url http://host:1234/v1 # no config at all
openlocal chat "hello" # one plain turn, no tools
openlocal tools # what the model can call
openlocal probe # measure native vs JSON tool calling
openlocal eval # 17 deterministic tasks, pass/fail
openlocal eval --via-mcp # same tasks, tools served over a REAL MCP server
openlocal leaderboard --repeat 2 # every configured model, one table
openlocal mcp # stdio MCP server
openlocal models / serve / stop / status / pull # local llama-server managementsmol is a shorter alias for the same CLI.
Your workspace is ~/.openlocal (or any directory containing models.toml, or
$OPENLOCAL_HOME). It holds models.toml, state.json, logs, downloads and eval results —
never the installed package.
[engine.ollama]
type = "external"
base_url = "http://127.0.0.1:11434/v1"
[[model]]
id = "qwen"
engine = "ollama"
family = "qwen" # tells openlocal this vendor's tool-calling quirks
served_model = "qwen3:4b" # the exact name the backend knows
default = trueenv | default | meaning |
|
| workspace directory |
| – | point every command at one |
| cwd | sandbox root for the file and shell tools |
| 8000 | hard cap on one tool result |
| dev commands | comma list, or |
| 120 | seconds before an MCP tool call is abandoned |
file_list(path=".") list files and folders
file_read(path, start_line=1, limit=200) read a file, one page at a time
file_write(path, content) write a file
file_edit(path, find, replace) replace text exactly, leave the rest alone
file_append(path, content) add to the end without touching what exists
file_grep(pattern, path=".") search files
web_search(query, limit=5) search the web (DuckDuckGo, or Tavily via key)
web_read(url, offset=0, max_chars=4000) read a page as text
shell_run(command, timeout=60) run one allowlisted command
calc(expression) exact arithmetic - models cannot count
finish(answer) agent loop only: end the taskHouse rules, enforced by tests: flat arguments with defaults, an example call in every
description, one-line result headers, hard-capped output with a truthful continuation hint,
and errors that name the next move — no such file: x - call file_list(".") to see the files that exist.
tools/ one registry → mcp_server.py (Claude Code speaks MCP to it)
→ agent.py (a local model calls the same tools)
_http.py the entire network layer, on urllib (this is why deps = 0)
runner.py external servers, or llama-server processes it starts and owns
protocol.py native tool_calls → loose JSON → repair → symptom-specific nudge
families.toml per-vendor quirks as DATA - the file contributors edit
evals/ 17 deterministic tasks + a leaderboard across modelsAdding a model family takes no Python: edit families.toml, add
your model to models.toml, then
openlocal probe --model your-model
openlocal eval --model your-modeland paste the resulting row into the PR. See CONTRIBUTING.md.
Running local GGUF models on AMD hardware (ROCmFP4 + MTP speculative decoding) is documented in docs/ENGINE.md — optional, and not needed for anything above.
MIT © DevXV3
This server cannot be deployed
Nifty's MCP server — exposes tasks, projects, messages, and files as tools for AI agents.
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
MCP-first toolbox for agents: KV storage, auth, queue, and utility tools. Free in early access.
