# woollama

**Web Over Ollama (and Llamas).** An MCP + OpenAI router for AI desktops.

📖 **Documentation: [woollama.readthedocs.io](https://woollama.readthedocs.io/)**

woollama sits between AI clients (Cursor, the OpenAI SDK, Claude Desktop,
cosmic-fabric, anything that speaks OpenAI or MCP) and AI backends (Ollama,
Anthropic, fabric, lackpy, filesystem MCPs, anything that speaks OpenAI or
MCP). It composes them into orchestrated calls without inventing a new
protocol.

```
                          ┌─────────────────────┐
                          │   AI clients        │
                          │   (any OpenAI or    │
                          │    MCP client)      │
                          └──────────┬──────────┘
                                     │
                  ┌──────────────────┴───────────────────┐
                  │            woollama                  │
                  │  OpenAI server  +  MCP server        │
                  │  ───────────────────────────────     │
                  │  routes models, tools, executors     │
                  │  composes patterns + tools + models  │
                  │  into named recipes                  │
                  └──────────────────┬───────────────────┘
                                     │
                  ┌──────────────────┴───────────────────┐
                  │                                      │
              ┌───┴────┐                            ┌────┴────┐
              │ MCP    │  tools, prompts, resources │ OpenAI  │  inference
              │ tool   │                            │ compat  │
              │ servers│                            │ backends│
              └────────┘                            └─────────┘
              fabric-mcp, lackpy,                   Ollama, Anthropic,
              filesystem, git, …                    vLLM, llama.cpp, …
```

## Status

**The Rust daemon `woollamad` — a multi-backend router, both surfaces live,
published to crates.io + PyPI.** woollama works end-to-end as:

- an **OpenAI-compatible server**: `/v1/chat/completions` (pass-through *and*
  hidden chat-loop orchestration of recipes, both with `stream:true` → OpenAI
  SSE), `/v1/models`, `/v1/tools`, and a **stateful surface** —
  `/v1/responses` + `/v1/conversations` (OpenAI Responses/Conversations shape;
  see below);
- an **MCP server** to its own clients — over **stdio** (`woollamad mcp`) and
  over **Streamable HTTP** at `/mcp`, mounted on the *same port* as `/v1/*`. It
  re-exports every discovered downstream tool (namespaced, with `output_schema`)
  plus a `chat` verb that emits live tool-progress notifications — i.e. it's an
  MCP aggregator.

It routes inference across **multiple backends** by `<provider>/<model>` —
`ollama` (local), `anthropic`, `openai`, `groq`, `together`, `openrouter`, and
**any OpenAI-compatible endpoint** you add in `inferencers.toml` (e.g.
self-hosted vLLM) — plus `claude-code/<model>`, a keyless path to Claude via the
local CLI (tool-less, or as an **executor** that runs a recipe's allow-listed
MCP tools itself — tool delegation). Config is file-driven (`mcp.json`,
`recipes.toml`, `inferencers.toml`).

**Stateful conversations** route *handles*; backends own the *state* — woollama
never stores transcripts in its own system. Two state-owning backends:
`claude-resume` (`claude --resume`, for `claude-code` models; keyless, the Claude
session owns the bytes) and `managed-agents` (Anthropic's Managed Agents, for
`claude-agent` models; `ANTHROPIC_API_KEY`, Anthropic hosts the session — and
exposes the transcript, so `/v1/conversations/{id}/items` works). Models with no
state-owning backend (ollama/cloud/recipe) are stateless — the caller owns
history (`store:false`). Long-lived MCP
connections. Served on **both a Unix socket** (`$XDG_RUNTIME_DIR/woollama.sock`,
mode 0600 — the default for local MCP clients) and an ephemeral loopback TCP
port; never `0.0.0.0` without explicit opt-in.

**Current status and what's next live in
[`docs/roadmap.md`](docs/roadmap.md).**

> **The Rust port is done (v0.5.x).** `woollamad` is the canonical router,
> published to crates.io (`cargo install woollama-server`) and PyPI
> (`pip install woollama`). The Python in `src/woollama/` is kept as the
> reference server and differential-test oracle — not deleted. See
> [`docs/rust-transition.md`](docs/rust-transition.md) for the (completed)
> transition criteria.

See `docs/architecture.md` for the full target design and
`docs/build-log.md` for the slice-by-slice history.

## Quick taste

The router is OpenAI-compatible, so any OpenAI client can drive it:

```python
import openai
c = openai.OpenAI(base_url="http://127.0.0.1:<port>/v1", api_key="x")

# Pass-through to Ollama
r = c.chat.completions.create(
    model="ollama/qwen3:14b-iq4xs",
    messages=[{"role": "user", "content": "Hi"}],
)

# Orchestrated: a recipe (system prompt + tools + model), transparent to the
# client. The chat-loop happens inside woollama; client sees only the final answer.
r = c.chat.completions.create(
    model="woollama/streamer",
    messages=[{"role": "user", "content": "Please count to 4."}],
)

# A device-aware inferencer (declares `management_url`): "default" resolves to
# whatever's currently loaded, and woollama loads/queues on demand — no
# not-loaded error, no wedge.
r = c.chat.completions.create(
    model="device/default",
    messages=[{"role": "user", "content": "Hi"}],
)

# Pass-through also covers images and embeddings:
img = c.images.generate(model="device/Z-Turbo", prompt="a small blue teapot")
vec = c.embeddings.create(model="device/Embedder", input="hello world")
```

woollama serves on **two transports at once**: a Unix socket at
`$XDG_RUNTIME_DIR/woollama.sock` (mode 0600 — the default for local MCP clients,
since a connectable socket can spend the router's API keys) and an ephemeral
loopback TCP port written to `$XDG_RUNTIME_DIR/woollama.addr` for clients to
discover. The `<port>` above is that ephemeral port. Same pattern as a local
`fabric --serve` instance.

## Install

The router is **`woollamad`** — a small Rust daemon. The Python implementation is
kept as a reference server and the differential-test oracle (see below), but
`woollamad` is the canonical router.

**From crates.io** (once published — `cargo install` ships only the binary, so
bring your own `mcp.json`):

```sh
cargo install woollama-server     # installs the `woollamad` binary
woollamad                         # starts the router; prints its address
```

**From this checkout** (works today; includes the bundled example MCP servers):

```sh
git clone https://github.com/teaguesterling/woollama
cd woollama
cargo build --release             # builds target/release/woollamad
./target/release/woollamad        # starts the router; prints its address
```

On startup `woollamad` prints its `OpenAI base_url` (e.g.
`http://127.0.0.1:<port>/v1`) — copy that into your OpenAI client. (It's also
written to `$XDG_RUNTIME_DIR/woollama.addr` for programmatic discovery, and it
serves the same surface over the `woollama.sock` unix socket.)

### Python 3.14: you will build from source, slowly

The `woollama` **Python** package depends on `woollama-core`, a compiled extension. We ship
wheels for **CPython 3.11–3.13 only** — pyo3 could not build against 3.14 until recently, and the
cp314 wheels are not published yet (#43).

Our metadata says `requires-python = ">=3.11"` with no upper bound, which is true — 3.14 works —
but on 3.14 there is no wheel, so the install falls back to **building the extension from
source**. That needs a Rust toolchain and takes minutes rather than seconds.

The trap is that `uv` picks an interpreter for you. `uv sync` in a project that merely allows
3.11+ will happily select 3.14 and then do a source build, or fail outright if no Rust toolchain
is present — and the error names `python-source` and maturin, neither of which mentions us. If
you want wheels today, pin the interpreter:

```sh
uv sync --python 3.12            # or any of 3.11-3.13
```

Any project depending on `woollama-core` **below 0.9.0** should pin 3.11–3.13 outright: those
versions cannot build on 3.14 at all, and every `woollama-core` sdist at or below 0.8.1 is
unbuildable on *any* Python (fixed in 0.8.2 — see the changelog for v0.14.4).

### The Python reference server

The original Python implementation still runs and is used as the live oracle that
keeps `woollamad` honest:

```sh
uv sync                           # creates .venv and installs deps
uv run woollama                   # the Python reference server
```

> **Prerequisite for the examples below:** they use `ollama/qwen3:14b-iq4xs`, so
> install [Ollama](https://ollama.ai), `ollama serve`, and
> `ollama pull qwen3:14b-iq4xs`. **No Ollama?** Use the keyless Claude path
> instead — `model="claude-code/haiku"` (needs the `claude` CLI logged in) — or
> any cloud model with its key set (see [Configuration](docs/configuration.md)).

### Tests & lint

```sh
# Rust (woollamad): the daemon's own suites
cargo test --tests --features test-fixtures
cargo build --release            # so the live oracle can spawn the binary

# Python: hermetic suite + lint
uv run --extra dev pytest        # hermetic suite (live tests are opt-in: -m integration)
uv run ruff check .              # lint — the CI gate

# The live differential oracle — same tests, against woollamad by default:
uv run --extra dev pytest -m integration            # targets target/release/woollamad
WOOLLAMA_TEST_CMD="python -m woollama" \
  uv run --extra dev pytest -m integration          # opt in to the Python reference
```

CI (`.github/workflows/ci.yml`) runs the Rust + Python gates on every push to `main` and PR.
For the same lint gate locally on commit, opt into the pre-commit hook:

```sh
uv tool install pre-commit && pre-commit install
```

Lint only — the project does not use `ruff format` (lines are hand-wrapped,
`E501` is ignored), so there is no formatter step in either gate.

## Design principles

1. **Two standards, neither extended.** MCP for tool/prompt/resource
   discovery and execution; OpenAI chat-completions for the inference
   primitive. woollama is a router between them.
2. **Local-only, ephemeral by default.** Random loopback port, persisted
   address file for discovery, never `0.0.0.0` without explicit opt-in — and
   the opt-in requires an auth token (`WOOLLAMA_TOKEN`; woollama refuses to
   start off-loopback without one). The router holds API keys and routes to
   local resources — it should not be LAN-reachable unauthenticated.
3. **The model namespace is the universal addressing scheme.** Raw inferencers
   (`<provider>/<model>`, e.g. `ollama/X`, `anthropic/X`, `claude-code/X`) and
   full recipes (`woollama/<recipe>`) are all addressable through OpenAI's
   standard `model` field. No new wire format.
4. **woollama owns routing, not inference or tools.** It uses other people's
   inference engines (Ollama, Anthropic, …) and other people's tool servers
   (any MCP server — filesystem, git, lackpy, …). It composes them.
5. **she talks to llamas.**

## What works today

- OpenAI surface: `/v1/models`, `/v1/chat/completions` (pass-through +
  recipe orchestration, both with `stream:true` → OpenAI SSE), plus
  `/v1/images/generations` and `/v1/embeddings` pass-through (see below).
  (`/v1/tools` introspection is Python-reference-only today — see
  [#23](https://github.com/teaguesterling/woollama/issues/23))
- **Stateful surface**: `/v1/responses` (stateless subset, incl. `stream:true` →
  OpenAI Responses SSE, + stateful) and `/v1/conversations` (create/list/get/
  delete, plus `items` where the backend exposes its transcript). woollama routes
  conversation *handles*; backends own state (woollama never stores transcripts
  itself) — `claude-resume` for `claude-code` models, `managed-agents` (Anthropic
  Managed Agents) for `claude-agent` models, with an interactive
  `requires_action` pause/answer path; models with no state-owning backend are
  stateless (`store:false`)
- Multi-backend routing by `<provider>/<model>`: ollama (incl. `num_ctx` honored
  via ollama's native `/api/chat`), anthropic, openai, groq, together,
  openrouter, `claude-code`, + any OpenAI-compatible endpoint via
  `inferencers.toml`
- **Tool delegation**: a `claude-code` recipe with tools runs as an *executor* —
  Claude owns the agentic loop and calls the recipe's allow-listed MCP tools
  itself (per-recipe `--mcp-config` + `--allowedTools` containment)
- **Image + embedding pass-through**: `POST /v1/images/generations` and
  `POST /v1/embeddings` forward a `<provider>/<model>` request to that
  inferencer's own OpenAI-compatible endpoints, mirroring the chat pass-through
  (non-streaming; unknown namespace → `400`)
- **Model pooling / device-aware inferencers**: an inferencer that declares a
  `management_url` gets on-demand model loading (no more bare not-loaded
  errors), stable **virtual model names** (`<provider>/default`, config
  `virtual` aliases), and per-model request queuing with `503` + `Retry-After`
  backpressure instead of a wedge — plus queue-aware LRU eviction to fit
  `pool_max` loaded models. Fully additive: an inferencer with no
  `management_url` is unaffected. Ships in **both** `woollama` (Python) and
  `woollamad` (Rust); pooling currently covers `/v1/chat/completions` only
  (`/v1/responses` isn't pooled yet)
- MCP server side: stdio (`woollamad mcp`) **and** Streamable HTTP at `/mcp` on
  the same port — recipes as **parameterized prompts** (their `{{var}}` tokens →
  arguments), a `chat` verb (with live tool-progress notifications), and every
  downstream tool re-exported with its `output_schema` (aggregator)
- **MCP client side, stdio *and* HTTP**: an `mcp.json` server is either a
  `command` (stdio subprocess) or a `url` (Streamable HTTP). Since woollama also
  *serves* `/mcp`, the `url` form is how **one woollamad consumes another** — an
  inference-holding instance reaching a tools-only instance's namespace, so the
  tools instance can hold zero provider API keys. Credentials are `${VAR}` in
  `headers`, validated fail-closed at load. `woollamad check-config` validates
  every config file and exits non-zero on any problem
- **Pattern templating** on woollama's own `/w1/` namespace (not OpenAI's `/v1/`):
  parameterized recipes/patterns with `{{var}}` substitution —
  `GET /w1/patterns` (discovery), `POST /w1/patterns/{name}/render` (assemble),
  `POST /w1/patterns/{name}/run` (render + infer, streaming). Patterns also come
  from a fabric-style directory scan (`[patterns]`) and a **fabric backend**:
  woollama can run/own `fabric --serve`, surface its library on `/w1/`, and
  transparently proxy fabric's API at `/fabric/*`. Pattern backends are pluggable
  (the `PatternBackend` trait — see `docs/extending.md`)
- File-driven config (`mcp.json`, `recipes.toml`, `inferencers.toml`), multi-
  MCP-server discovery + unified tool registry, long-lived MCP connections
- Recipe allow-list enforced as a security boundary (in-loop AND in delegation);
  served on a **Unix socket + loopback TCP**, address discovery file; CI
  (ruff + hermetic suite, 3.11/3.12)

## Not yet (next on the roadmap)

- The live, interactive Claude-in-tmux session backend (a separate Rust session
  driver) — gated on spikes that need a real terminal. (The interactive
  `requires_action` path itself already works via the managed-agents backend.)
- cosmic-fabric actually consuming the conversations surface (the last open
  integration milestone). The generic `store-backed` mechanism + two reference
  store providers (MCP + REST) already ship; what's pending is the cross-repo
  wiring. (Pattern templating + the fabric backend it needed have **shipped** —
  see `docs/patterns.md`.)
- lackpy re-pinning to the now-published `woollama-core` wheel.

Full scorecard, ordering, and pending verifications:
**[`docs/roadmap.md`](docs/roadmap.md)**.

## Origin

woollama is the production-grade rewrite of an architecture co-designed
in [cosmic-fabric](https://github.com/teaguesterling/cosmic-fabric), which
remains a frontend (and will use woollama as its router engine). The design
docs that brought woollama here:

- `docs/architecture.md` — the model/tool/executor router design
- `docs/naming.md` — how we landed on this name

## License

MIT — see `LICENSE`.