<div align="center">

# openlocal

### Open tools. Local brains. Zero cloud.

**A tool belt that small local models can actually use — and the receipts to prove it.**

[![CI](https://github.com/DevXV3/openlocal/actions/workflows/ci.yml/badge.svg)](https://github.com/DevXV3/openlocal/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/openlocal.svg)](https://pypi.org/project/openlocal/)
[![Python](https://img.shields.io/pypi/pyversions/openlocal.svg)](https://pypi.org/project/openlocal/)
[![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen)](pyproject.toml)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

</div>

---

```bash
pip install openlocal
openlocal quickstart
```

That's it. `quickstart` finds the model server you already run — **ollama, LM Studio,
llama.cpp, vLLM** — writes your config, measures which tool-calling protocol your model can
actually drive, and runs a real task so you see it work:

```text
$ openlocal quickstart
openlocal quickstart - looking for a model server...
found ollama at http://127.0.0.1:11434/v1 with 6 model(s)
wrote ~/.openlocal/models.toml
default model: qwen3-4b  (served as qwen3:4b)
measuring which tool protocol this model can drive...
  -> native tool calling

demo: Read notes.md and tell me how many lines it has.
  tool     file_read
  final

  The file notes.md has 3 lines.

You are set up. Try:
  openlocal run "list the files here and summarise what this project is"
  openlocal eval          # score this model on 15 deterministic tool tasks
  openlocal mcp           # serve these tools to Claude Code over MCP
```

## Why openlocal

| | |
|---|---|
| 🪶 **Zero dependencies** | Pure standard library. Installs in seconds, runs on a Raspberry Pi, and can't be broken by somebody else's release. |
| 🔌 **Works with what you have** | Anything that speaks the OpenAI `/v1` API. One URL, no adapters, no accounts. |
| 📊 **Measured, not claimed** | 10 models from 9 vendors, scored on 17 deterministic tasks. No LLM judge — every check is a file diff or a regex. |
| 🧰 **One registry, two surfaces** | The same 8 tools power an MCP server (Claude Code, any MCP client) and a local-model agent loop. |
| 🩹 **Tuned on real failures** | Every ergonomic rule here was paid for by a model breaking. The [lessons table](#why-your-small-model-fails-at-tools) is the part you can't get anywhere else. |
| 🧩 **Add a vendor by editing TOML** | New model family? Edit `families.toml`, run `openlocal probe`, send a PR. No Python needed. |

## The leaderboard

Same tasks, same tools, one model at a time, two runs each. `tools` is the protocol each
model was **measured** driving — not the one its docs claim.

| model | vendor | tools | passed | pass rate | bad json | tok/s |
|---|---|---|---:|---:|---:|---:|
| `qwen3.8-4b` | Alibaba | native | **17/17** | 100% | 0 | 40.4 |
| `glm-4.7-flash` | z.ai | native | **17/17** | 100% | 0 | 39.9 |
| `qwen3.8-27b` | Alibaba | native | **17/17** | 100% | 0 | 16.6 |
| `nemotron-nano-4b` | NVIDIA | native | **16/17** | 94% | 0 | 54.9 |
| `gemma-4-e4b` | Google | native | **16/17** | 94% | 0 | 50.3 |
| `gpt-oss-20b` | OpenAI | native | **16/17** | 94% | 2¹ | 45.3 |
| `granite-4.1-3b` | IBM | native | **15/17** | 88% | 4 | 66.0 |
| `llama-3.1-8b` | Meta | native | **14/17** | 85% | 2 | 33.4 |
| `mistral-small-3.2` | Mistral | native | **14/17** | 82% | 0 | 11.1 |
| `phi-4-mini` | Microsoft | native | **13/17** | 79% | 0 | 61.5 |

<sub>¹ gpt-oss's harmony parser rejects some of its own tool calls (llama.cpp answers 500).
The client's retry ladder recovers every one — the `retries` column in
[`RESULTS.md`](openlocal/evals/RESULTS.md) keeps that honest.</sub>

Reproduce it: `openlocal leaderboard --repeat 2`

## Why your small model fails at tools

A 3B model doesn't fail because it's stupid. It fails because tool output is a prompt, and
most tool output is written for humans. Everything below was **measured** — kept when the
score went up, reverted when it didn't.

| what broke | the fix | measured effect |
|---|---|---|
| "What is line 300?" → model answered line 311 | number every line: `300\| row 300` | fixed in one turn, across models |
| Model walked a 500-line file one line per turn | footers state facts, never commands ("file continues" ≠ "call again with…") | phi: 12 wasted steps → 2 |
| Prose instead of a tool call | prose with no tool = the final answer (native only) | phi: 51 malformed replies → 5 |
| …but "Sure, I'll use file_grep" is a **plan** | intent detection: plans get nudged, answers get accepted | runs stopped ending on intentions |
| `file_read(notes.md)` typed as text | prose call-syntax is parsed and executed | wasted turns eliminated |
| `{"limit": None}`, trailing junk, split objects | JSON repair: brace scan, tail closing, Python literals, fragment merge | phi bad_json 5 → 0 |
| Model wrote the file **before** reading the source | stale-write detection at finish time | llama: 13 → 15 / 15 |
| Model guessed "44" and repeated it when nudged | one turn of `tool_choice: "required"` | granite recovered the task |
| Read a 4-line CSV perfectly, said the sum was 40 | a `calc` tool — and a guard so it can't be used *before* reading | granite: +2 tasks |
| Wrote the output file without ever reading the source | thin-air-write detection at finish time | qwen-4b over MCP recovered |
| Copied the `N\|` reference prefixes into edit targets | `file_edit` strips them when the raw text does not match | edit_code: 10/10 models pass |
| Same call, forever (A→B→A→B) | repeat guard replays the cached result | llama pagination loop gone |
| Server 500s on its own tool-call syntax | retry ladder: as-asked → warmer → same turn without `tools` | gpt-oss: 11/15 → 14/15 |
| **Reverted:** one more system-prompt rule | — | granite 12/15 → **11/15**. Fewer rules win. |

## MCP is a first-class citizen, not a wrapper

`openlocal eval --via-mcp` reruns the whole eval suite with the tools served over a real
stdio MCP server — schemas crossing the wire, results as content blocks, a subprocess per
task. The scores match the in-process numbers, which is the point: what Claude Code
experiences is what was measured.

## Use it from Claude Code (or any MCP client)

```bash
pip install "openlocal[mcp]"
claude mcp add openlocal -- openlocal mcp
```

The same 10 tools your local model uses, now in Claude Code. `finish` stays behind — it's
loop control, not a capability.

## Commands

```bash
openlocal quickstart              # find a server, configure, probe, demo
openlocal run "goal"              # agent loop over your tools
openlocal run "goal" --url http://host:1234/v1     # no config at all
openlocal chat "hello"            # one plain turn, no tools
openlocal tools                   # what the model can call
openlocal probe                   # measure native vs JSON tool calling
openlocal eval                    # 17 deterministic tasks, pass/fail
openlocal eval --via-mcp          # same tasks, tools served over a REAL MCP server
openlocal leaderboard --repeat 2  # every configured model, one table
openlocal mcp                     # stdio MCP server
openlocal models / serve / stop / status / pull    # local llama-server management
```

`smol` is a shorter alias for the same CLI.

## Configuration

Your workspace is `~/.openlocal` (or any directory containing `models.toml`, or
`$OPENLOCAL_HOME`). It holds `models.toml`, `state.json`, logs, downloads and eval results —
never the installed package.

```toml
[engine.ollama]
type = "external"
base_url = "http://127.0.0.1:11434/v1"

[[model]]
id = "qwen"
engine = "ollama"
family = "qwen"              # tells openlocal this vendor's tool-calling quirks
served_model = "qwen3:4b"    # the exact name the backend knows
default = true
```

| env | default | meaning |
|---|---|---|
| `OPENLOCAL_HOME` | `~/.openlocal` | workspace directory |
| `OPENLOCAL_BASE_URL` | – | point every command at one `/v1` server |
| `SMOL_ROOT` | cwd | sandbox root for the file and shell tools |
| `SMOL_MAX_CHARS` | 8000 | hard cap on one tool result |
| `SMOL_SHELL_ALLOW` | dev commands | comma list, or `*` |
| `SMOL_TOOL_TIMEOUT` | 120 | seconds before an MCP tool call is abandoned |

## The tools

```text
file_list(path=".")                       list files and folders
file_read(path, start_line=1, limit=200)  read a file, one page at a time
file_write(path, content)                 write a file
file_edit(path, find, replace)            replace text exactly, leave the rest alone
file_append(path, content)                add to the end without touching what exists
file_grep(pattern, path=".")              search files
web_search(query, limit=5)                search the web (DuckDuckGo, or Tavily via key)
web_read(url, offset=0, max_chars=4000)   read a page as text
shell_run(command, timeout=60)            run one allowlisted command
calc(expression)                          exact arithmetic - models cannot count
finish(answer)                            agent loop only: end the task
```

House rules, enforced by tests: flat arguments with defaults, an example call in every
description, one-line result headers, hard-capped output with a truthful continuation hint,
and **errors that name the next move** — `no such file: x - call file_list(".") to see the
files that exist`.

## Architecture

```text
tools/          one registry  →  mcp_server.py   (Claude Code speaks MCP to it)
                              →  agent.py        (a local model calls the same tools)
_http.py        the entire network layer, on urllib (this is why deps = 0)
runner.py       external servers, or llama-server processes it starts and owns
protocol.py     native tool_calls → loose JSON → repair → symptom-specific nudge
families.toml   per-vendor quirks as DATA - the file contributors edit
evals/          17 deterministic tasks + a leaderboard across models
```

## Contributing

Adding a model family takes no Python: edit [`families.toml`](openlocal/families.toml), add
your model to `models.toml`, then

```bash
openlocal probe --model your-model
openlocal eval  --model your-model
```

and paste the resulting row into the PR. See [CONTRIBUTING.md](CONTRIBUTING.md).

Running local GGUF models on AMD hardware (ROCmFP4 + MTP speculative decoding) is documented
in [docs/ENGINE.md](docs/ENGINE.md) — optional, and not needed for anything above.

## License

MIT © DevXV3