willow-mcp

Agent-neutral MCP server with persistent memory and task execution. Works with any MCP client: Claude Code, Claude Desktop, Cursor, or any custom agent that speaks stdio MCP.

Three storage backends in one server:

• SOIL store — SQLite-backed local key/value store with full-text search and soft delete

• Postgres knowledge base — multi-keyword searchable knowledge graph

• Kart task queue — sandboxed task executor for shell commands and scripts

Every tool call is authorized via a filesystem-based manifest ACL — no ACL database, no external auth service. See Authorization.

Install

pip install willow-mcp

Requires Python 3.11+. Postgres is optional — SOIL store works standalone.

willow-mcp-init    # scaffold $WILLOW_HOME (idempotent)
willow-mcp-compile --force   # compile manifests (use product venv — see below)
willow-mcp-sign-seed hanuman # ratify home seed + detach-sign (operator terminal only)
willow-mcp-compile-persona hanuman # seed → personas/hanuman.md (AS-7)

PATH note: ~/.local/bin/willow-mcp is often the fleet shim (sap_mcp.py), not this product. Use the product venv binary: ~/github/.willow/venvs/willow-mcp/bin/willow-mcp-compile --force or .../bin/willow-mcp compile-agents --force after pip install -e . in that venv.

Runtime layout: docs/design/product-layout.md (LOCKED).

Related MCP server: Hoard

Tools

Egress needs three keys

A task that reaches the network requires all three of these, and any one missing denies before anything is written:

// $WILLOW_HOME/settings.global.json — the off switch
{ "consent": { "internet": false, "cloud_llm": true, "lan": false } }

$ willow-mcp grant-net myapp --ttl 30m --reason "publish the release"
$ willow-mcp net-status          # what is live, and for how much longer
$ willow-mcp revoke-net myapp    # or just wait; the ceiling is 3h

Setting consent.internet to false stops network tasks submitted through task_submit, immediately, without editing a single manifest. task_net is a capability (rarely granted, deliberately excluded from full_access); consent.internet is a switch; the lease is a time-boxed grant that an agent may ask for and never issue. No MCP tool can mint one — grant-net is local CLI only, exactly like confirm-binding. An agent may request egress and may never grant it to itself.

It is not a fleet-wide kill switch, and this README used to say it was. The three keys gate the submitter. The executor — kartikeya, and willow-2.0's own Kart copy — reads # allow_net out of the task text and honors it on sight; consent, lease, and task_net appear nowhere in it. The tasks table is shared Postgres, so any other submitter that writes a row carrying that directive reaches the network regardless of this file (B-37, P0, verified live with task_net revoked and the lease expired). Until B-37 lands, consent.internet: false is an off switch for one door in a building with more than one door.

Consent and leases are both read fail-closed: a missing file, an unparseable file, a non-boolean value ("true", 1), a lease past its deadline, a deadline with no timezone, or a lease record naming a different app than the file it sits in — all read as denied. Absence is not consent, and a name is not an identity. willow-mcp only reads the consent policy — it is authored by willow-2.0's global_settings.py. That module also keeps a flat consent.json mirror, rewritten on every save; willow-mcp reads it only when the canonical file is absent. Because it is written constantly and read almost never, it can drift silently, and deleting it does not keep it gone — the next save recreates it. If the two disagree, diagnostic_summary reports an error naming both values rather than quietly obeying one of them (B-30).

gates — every gate, on/off, egress-lease shaped

Diagnosing a denial today means knowing which of a dozen-plus gates to check and which file or CLI command controls it. willow-mcp gates shows all of them at once, each rendered the way the egress lease already renders itself — on/off, plus how long the "on" is good for. Run it in a real terminal and it's interactive — arrow keys / j-k to move, enter/space to actually flip the highlighted gate, no second command to copy anywhere:

$ willow-mcp gates                    # interactive TUI (every app under mcp_apps/)
$ willow-mcp gates myapp              # interactive TUI, scoped to one app
$ willow-mcp gates --serve            # live local HTML dashboard, working buttons
$ willow-mcp gates --serve --port 9000 --host 127.0.0.1
$ willow-mcp gates --static           # one-shot text printout instead of the TUI
$ willow-mcp gates --html             # writes ./willow-gates.html, a read-only snapshot
$ willow-mcp gates --json             # raw rows, for scripting

--static/--json/--html are unchanged from before and still the right choice for scripting, CI, or a file you want to keep — --static is also what runs automatically whenever stdout isn't a real terminal (piped, redirected), so nothing here breaks existing scripts.

The interactive TUI and --serve's live dashboard share one action layer (gates_actions.py) with the CLI subcommands below — pressing a row (or clicking its button) calls the exact same functions allow-permission/ grant-net/confirm-binding do, nothing new. --serve binds 127.0.0.1-only by default; it's a mutation-capable local admin surface with no authentication of its own, so widening --host prints a warning rather than doing it quietly. The one exception is the worker row's action: it drains the queue once (like worker --once), never launches the persistent daemon — that would block the TUI/dashboard forever.

Manifest permission groups — which had no CLI before, only hand-editing manifest.json or regenerating it via compile-agents — get their own pair, usable standalone or as what the TUI/dashboard call underneath:

$ willow-mcp allow-permission myapp store_read
$ willow-mcp deny-permission myapp store_read

Both are local-CLI-only, never MCP tools, for the same reason grant-net isn't: an agent must never be able to grant itself a permission it was just denied — and that boundary holds for the TUI and --serve too, since neither is reachable except by an operator running them on the host that owns $WILLOW_HOME. consent.* rows never show a command or a working button — willow-mcp only reads that policy (see above) — and strict_trust_root / severance / human-orchestrator attestation are environment variables read once at process start, so their rows name the env var to set and restart with, rather than pretending a live toggle exists.

task_net and integration_net both show up as their own capability rows (neither is folded into full_access), and both are authorized by the same per-app egress lease below them — one grant-net/revoke-net covers Kart sandbox egress and server-process integration calls together, since a lease is scoped to the app, not to which capability is asking.

Every row also carries a state_label in context instead of a bare ON/OFF — GRANTED, ALLOWED, ACTIVE, CONFIRMED, RUNNING, ENABLED (and their opposites) — and a category (egress & network / system / identity / permissions) that the TUI and both HTML pages group by. The HTML pages open on the egress tab — the smallest group, and the one with a clock — with a summary strip above the tabs for at-a-glance state, and render the ~20-row permissions group (routine, rarely touched) as a compact list rather than large cards, instead of one flat scroll of everything at once.

tree — the integration seam for a real dashboard

docs/design/*.html sketches a client UI as a tree — trunk (overall health), sap (task queue), canopy (agent fleet), roots (SOIL store), rings (schema-mapping confirmation), leaves (knowledge atoms), litter (activity log), and stomata (the gates above). willow-mcp tree is what makes that real: one call that returns every part in that same shape, instead of a dashboard assembling fleet_status/fleet_health/kb_startup_continuity/ receipts_tail/gates itself.

$ willow-mcp tree myapp              # short text summary
$ willow-mcp tree myapp --json       # full data, for a real dashboard to consume

It's a thin CLI wrapper over willow_mcp.tree_view.build_tree(app_id), which a Python dashboard can also import and call directly. sap, canopy, and leaves go through the same @_guarded MCP tool functions a client would reach over the protocol — gating, rate limiting, and receipt logging all still apply — and degrade to {"error": "postgres_unavailable"} with no database configured, same as those tools already do. roots, rings, litter, and stomata read local SQLite/filesystem state directly, so they work with no Postgres at all.

The residual, stated plainly

On a host where the agent and the MCP server run as the same uid, the agent can write the very files that authorize its egress. Leases make a self-grant expire and leave a record, and the PreToolUse hook blocks the obvious attempts — but the operating system is not stopping it. diagnostic_summary names exactly which keys the running process could forge, under checks.net_lease.self_writable.

The control is ownership. Put mcp_apps/ and mcp_apps/_net_leases/ under a uid the agent does not run as, then:

$ export WILLOW_MCP_STRICT_TRUST_ROOT=1   # refuse egress when the keys are self-writable

Strict mode is off by default because turning it on before that separation exists would deny egress on every current install. This is tracked as B-32 in docs/BUGS.md; requesting egress and confirming it are separate authorities, and until the filesystem says so, only convention does.

Integrations (outbound adapters)

integration_call lets the server process call external HTTP APIs through registered adapters — a second egress lane, beside the Kart sandbox's. It uses the same three-key gate, but keyed on its own capability, integration_net: the server egresses as its own uid with its own filesystem view, a strictly more privileged lane than the network-namespaced sandbox, so task_net never implies it (and vice versa). integration_call itself is also excluded from full_access — even the attempt surface is opt-in.

Adapters are earned, not scaffolded. Two are live (github, huggingface); six are declared stubs (gmail, slack, notion, google-drive, datadog, jira) that refuse fail-closed, each naming what it needs and what earns its implementation. integration_list is the ledger — see docs/design/integrations.md for the earn rule.

Credentials resolve environment-variable-first (e.g. WILLOW_GITHUB_TOKEN, then GITHUB_TOKEN), then the vault under integration/<name>/token. No tool ever returns a credential — only its source.

$ willow-mcp-integrations list                # the ledger, live + stubs
$ willow-mcp-integrations check github --app-id myapp   # offline: creds? keys? no network call
$ willow-mcp-integrations set-token github   # prompted + hidden, stored in the vault

Running the task worker

task_submit only queues a task. A worker process executes it, sandboxed with bubblewrap. Without one running, tasks stay pending forever:

willow-mcp worker --lane fast     # daemon; polls until stopped
willow-mcp worker --once          # drain what's queued, then exit

The engine is kartikeya, a hard dependency — a base pip install willow-mcp ships a working drainer.

A running worker publishes a heartbeat under $WILLOW_HOME/worker_heartbeat/, which fleet_health reads back:

{"pending": 3, "running": 0, "completed": 12, "failed": 0, "total": 15,
 "workers": {"alive": 0, "workers": [{"pid": 4242, "state": "dead", ...}]},
 "stranded": true}

stranded: true means there is pending work and no live worker — the distinction between "queued, it'll run" and "queued, nothing is listening." diagnostic_summary raises the same condition as a named worker problem. A worker is alive (ticking), stale (process up, loop wedged), or dead (pid gone). Heartbeats are advisory telemetry: no permission decision reads them, and reads verify the recorded pid is a live local process, so a forged file naming a dead pid reads dead.

knowledge_search/kb_at/kb_startup_continuity and fleet_status adapt to whatever your host database's real columns are named — see docs/design/schema-adaptation.md. knowledge_ingest/kb_ingest/kb_journal/kb_promote refuse to write (unconfirmed_schema) until you've reviewed and confirmed that mapping via schema_confirm_mapping — the schema-confirm skill walks through that.

Every tool requires an app_id param, checked against a manifest at $WILLOW_HOME/mcp_apps/<app_id>/manifest.json — see Authorization. The one exception is diagnostic_summary, which is intentionally ungated: it is the tool you reach for when your manifest or database is misconfigured, so gating it behind a permission would make the diagnostic itself undiagnosable. It discloses only the caller's own configuration — never fleet rows or vault secrets — and in serve mode still requires a confirmed identity and redacts absolute filesystem paths.

MCP config

Repo-local configs (.cursor/mcp.json, .mcp.json) wire willow-mcp plus codebase-memory-mcp for graph-augmented code search while developing this package. Install the CBM binary to ~/.local/bin/codebase-memory-mcp, then index this repo (project: home-sean-campbell-github-willow-mcp).

willow-mcp's entry points at a repo-local venv rather than a bare python3 — your host interpreter may not have pip or the mcp package installed (a missing import here crashes the stdio server before the handshake, which shows up as a client-side reconnect failure). Set it up once per clone:

python3 -m venv .venv
.venv/bin/python3 -m pip install -e .

Minimal single-server config (path is relative to the repo root, so this works unmodified on any clone once the venv above exists):

{
  "mcpServers": {
    "willow-mcp": {
      "type": "stdio",
      "command": ".venv/bin/python3",
      "args": ["-m", "willow_mcp"]
    },
    "codebase-memory-mcp": {
      "type": "stdio",
      "command": "codebase-memory-mcp",
      "args": []
    }
  }
}

Point WILLOW_PG_DB / WILLOW_STORE_ROOT at your host fleet store when you need Postgres knowledge or shared SOIL data.

You can also run the full willow-2.0 server directly — the tool API is identical, apps work against both transparently.

HTTP serve mode (OAuth)

Serve mode is 2.0.0+. Until the 2.0.0 release lands on PyPI, install from source (pip install -e . in a clone) to use it.

Beyond stdio, willow-mcp can run as an HTTP server that authenticates callers with OAuth 2.0 + PKCE against Google or Apple as the upstream identity provider. Signing in proves who a caller is; a separate, operator-controlled identity binding step maps that identity to an app_id before any tool permission applies. An authenticated-but-unbound caller is denied exactly like an unmanifested app_id — fail closed, never fail open.

1. Store provider credentials in the local vault (secrets are prompted, so they never land in shell history or a process listing):

willow-mcp setup --google-client-id "<client-id>"        # prompts for the secret
# or, for Apple:
willow-mcp setup --apple-team-id "<team>" --apple-client-id "<svc>" \
                 --apple-key-id "<kid>" --apple-p8-key-path ./AuthKey.p8

2. Run the server:

python3 -m willow_mcp --serve --port 8765 --host 127.0.0.1

--port/--host take precedence over WILLOW_MCP_PORT/WILLOW_MCP_HOST, which take precedence over the defaults (8765 / 127.0.0.1). Point an HTTP MCP client at http://<host>:<port>/mcp.

3. First sign-in proposes a binding. When a person completes the Google/Apple approval flow, the server writes an unconfirmed binding to $WILLOW_HOME/mcp_apps/_identity_bindings/<issuer>__<subject>.json:

{ "issuer": "google", "subject_id": "…", "email": "you@example.com",
  "email_basis": "asserted", "app_id": null, "confirmed": false }

email_basis records how much downstream code should trust the email, because IdPs differ: asserted (Google — present and IdP-asserted every sign-in), first_auth_only (Apple — may appear only on the first authorization), relay (Apple private-relay address that can stop forwarding), or unavailable. If a bound identity's email later changes between sign-ins, the binding is annotated with email_drift rather than silently updated.

4. Confirm the binding (operator-only, local). Confirmation is deliberately not an MCP tool — a remote caller must never confirm its own binding. Run it on the host that owns $WILLOW_HOME:

willow-mcp confirm-binding --issuer google --subject "<subject-id>" --app-id "<app_id>"

Only after this does the caller's session resolve to the manifest permissions for <app_id> (see Authorization).

Turning serve mode on and off

Serve mode is a background process, not part of the stdio server — so it's turned on and off on demand rather than by editing config each time. scripts/willow-serve manages a systemd --user service for the --serve process and toggles the matching http entry in .mcp.json, so an MCP client connects to it only while it's on:

scripts/willow-serve install   # one-time: write + load the systemd user unit
scripts/willow-serve on         # start serve + add the .mcp.json entry
scripts/willow-serve off        # stop serve  + remove the .mcp.json entry
scripts/willow-serve status     # unit state + whether the entry is present
scripts/willow-serve logs        # follow the serve logs (journalctl)

After on/off, reconnect your MCP client (in Claude Code: /mcp) so it picks up the changed .mcp.json. Port/host default to 8766/127.0.0.1; set WILLOW_MCP_PORT / WILLOW_MCP_HOST before install to change them. Claude Code users get this as the willow-serve skill — just ask to turn serve mode on or off.

If you already signed in once, on reuses your cached credential — no OAuth screen reappears unless it was cleared. That's expected, not a failure.

Serve mode does not inherit your shell environment. The systemd --user unit is started by systemd, not by your interactive shell, so a WILLOW_PG_DB (or WILLOW_STORE_ROOT, WILLOW_HOME, …) you export in .bashrc/.zshrc will not reach the serve process — it falls back to the defaults in the Configuration table. This bites env-based, non-default setups: the stdio server (launched from your shell) reads willow_20, say, while serve silently reads the default willow. Make the config reachable by the unit before on:

# one-time: import current shell values into the systemd --user manager
systemctl --user import-environment WILLOW_PG_DB WILLOW_STORE_ROOT WILLOW_HOME
# …or, durably, drop them in a file systemd --user reads at login:
#   ~/.config/environment.d/willow-mcp.conf  →  WILLOW_PG_DB=willow_20

Then scripts/willow-serve install (regenerate) and on. Verify with a read tool over the serve endpoint: a table_not_found / relation … does not exist on data that stdio can see is the signature of this env gap.

Configuration

Authorization

Manifest-based ACL, no external service or ACL database. Each app_id needs a manifest at $WILLOW_HOME/mcp_apps/<app_id>/manifest.json:

{"permissions": ["store_read", "knowledge_write"]}

permissions is a list of group names and/or literal tool names — see PERMISSION_GROUPS in src/willow_mcp/gate.py for the full set (store_read, store_write, knowledge_read, knowledge_write, schema_admin, task_queue, agent_dispatch, fleet_read, context, audit, gap_read, gap_write, gap_promote, full_access). Fail-closed: no manifest, or an empty permissions list, denies every call for that app_id. gap_promote is kept separate from gap_write — landing something as trusted knowledge is a more consequential act than logging or resolving a gap, the same reasoning schema_admin gets its own group instead of folding into knowledge_write.

There is also one capability permission, task_net, which is not a tool name but a privilege flag: it lets an app ask for task_submit(allow_net=True). It is deliberately excluded from task_queue and full_access — network egress from the sandbox must be granted explicitly, on its own line, and only host-side (never authored from inside the sandbox). On its own it authorizes nothing: the call also needs the operator's consent.internet and a live egress lease (see Egress needs three keys).

store_scope — confining an app to its own collections

By default, store_* tools are unrestricted across collections — and by default the SOIL store is the wider Willow fleet's store (see WILLOW_STORE_ROOT in Configuration above), so an app with store_read/store_write/full_access can see every collection any other app or fleet process has written, the same way it always could. That's the right default for a single-operator, single-trust-domain install, but it means a store_read grant to one app is implicitly a grant to read every other app's data too.

Sharing is a default, not a design commitment. An install that should be cut off from the fleet can point WILLOW_STORE_ROOT at its own store and name the fleet it is severed from — see Severance below, which turns the cut into something diagnostic_summary checks rather than something the docs assert.

An operator who wants an app confined to its own data adds an optional store_scope array to that app's manifest:

{"permissions": ["full_access"], "store_scope": ["myapp_*"]}

Patterns match by exact name, or by prefix if they end in *. With store_scope set, store_put/get/list/update/search/delete reject any collection outside it (collection_denied), and store_search_all only searches the matching collections instead of every collection in the store. Omit the field entirely for today's unrestricted behavior — an empty list ("store_scope": []) means "no collections," not "unrestricted."

A scope the gate cannot read denies everything. If store_scope is present but malformed — most likely "store_scope": "myapp_*", a string where a list belongs — the app is confined to no collections rather than granted all of them. The same holds for an unreadable manifest or an invalid app_id. This is deliberate: an operator who mistypes the field believes the app is confined, and a policy that cannot be parsed is not consent. The app fails loudly, an ERROR is logged naming the field and the type it got, and nothing leaks while the typo is being found. Omit the field (or set it to null) to declare no policy.

In HTTP serve mode, the app_id is not taken from the call — it is resolved from the caller's confirmed OAuth identity binding, then checked against that same manifest ACL.

Severance

A willow-mcp install can share a Willow fleet's store, database, and trust root, or it can be cut off from them. Both are legitimate. What is not legitimate is claiming the cut and not having it — a server that reports ok while wired to the fleet is worse than one with no check at all.

Severance is asserted, never assumed. Name the fleet you are severed from:

export WILLOW_MCP_FLEET_HOME=/home/you/github/.willow
export WILLOW_MCP_FLEET_PG_DB=willow_20

diagnostic_summary then reports a severance check over three surfaces:

The distinction is the whole design. Store and database hold data: someone who writes them corrupts records. mcp_apps/ holds authority — the manifest that grants task_net, the lease root, the consent file. Someone who writes those grants themselves the egress the cut was supposed to deny. Only the third can turn a severed install into a compromised one, so only it breaks the verdict.

Consequently the trust root must live somewhere neither this process nor the Kart sandbox can write. A repo directory is the wrong place for it, however convenient: repos are bound read-write into task sandboxes. Put data in the repo; put the gate outside it, owned by a uid the agent does not run as.

Symlinks are resolved before comparison. ~/.willow is frequently a symlink into a fleet tree, and two names for one directory are not two directories.

Leave both variables unset and the check reports not_asserted and changes nothing — a single-trust-domain install is complete without severance, and one that never claimed to be cut off cannot be caught lying about it. Set one and not the other and the unnamed surface reports unknown, which degrades: an unverifiable claim is not a passing one.

Hooks and skills (Claude Code)

.claude-plugin/plugin.json registers a PreToolUse hook and a skill for Claude Code users — install this package as a plugin to get both alongside the MCP server itself:

• hooks/pre_tool_use.py blocks Bash commands that reach for raw psql/psycopg2/sqlite3 against a database or store willow-mcp owns, redirecting to the matching MCP tool instead. It also blocks any call that would write the keys authorizing the agent's own egress — minting a lease, running grant-net, or editing a manifest to add task_net — and warns on a task_submit that hand-embeds a # allow_net directive.

• skills/schema-confirm.md walks through reviewing and confirming a table's schema mapping before writing to it.

• skills/willow-serve.md turns OAuth serve mode on/off on request (see above).

• skills/kart-tasks.md covers submitting and polling Kart tasks, the three-key egress model, and worker liveness.

See docs/design/hooks-and-skills.md for the design and the reasoning behind shipping these alongside tools rather than as a later add-on.

License

MIT — Sean Campbell 2026