herdr-voice-mcp

An MCP server that lets you run a team of coding agents by voice. You talk to Claude on your phone, and Claude hands out work to the Claude Code and Codex sessions on your desk machine, checks on them and reads back what they did.

It connects Claude to Herdr, a terminal multiplexer for coding agents, through Herdr's local socket API. A small companion service pushes a notification to your phone when an agent finishes or needs a decision.

Status: early, but used daily by the author. Single-user by design. Not affiliated with Herdr or Anthropic.

What it feels like

You are out for a walk with earbuds in and the Claude app in voice mode.

• "How are things?" Claude calls status and tells you that two agents are working, one is done and one is waiting for approval.

• "Let's do a stand-up." Claude calls standup and narrates who finished, who has been stuck and for how long, with the last lines each agent printed.

• "Tell the web agent to run the tests and fix what fails." Claude calls send. The server confirms delivery right away. The agent keeps working on your machine.

• Your phone buzzes: "Acme Web App: finished". You ask "What's new?", and Claude reads it back.

Related MCP server: noisy-coding

How it works

You (voice) → Claude app → Claude (Anthropic cloud)
                 → HTTPS → herdr-voice-mcp (your machine)
                 → Unix socket, JSON lines → Herdr → agent panes (Claude Code, Codex, …)

herdr-voice-notify (your machine) → Herdr event stream → ntfy / Pushover → your phone

Claude does not run on your phone or on your machine. It runs in the cloud, so the MCP server needs a public, authenticated HTTPS address even when you sit next to it. At your desk you can also use it locally from Claude Code over stdio, without any network exposure.

Claude only answers when you speak. It cannot tell you on its own that an agent finished. That is what the notifier is for: the phone gives you the signal, Claude gives you the content.

Tools

Targets are resolved leniently: agent name, pane ID (w4:p1), project key, name or alias, workspace label, tab label or part of the session topic. If a target is ambiguous, Claude gets the list of candidates and asks back.

Herdr states: working, blocked (a question or approval dialog is on screen), done (finished and not yet looked at in Herdr), idle (ready) and unknown.

Requirements

• macOS or Linux. The launchd templates are macOS-only. The server itself is plain Node and runs anywhere Herdr runs.

• Node.js 22 or newer.

• Herdr 0.9 or newer (socket protocol 22), running as a server with your agents inside it. Check with herdr status server.

• A Claude client that supports remote MCP servers: the Claude app (custom connectors) for mobile use, or Claude Code for local use.

• A way to reach the server from the internet, only for mobile use. The tested path is Tailscale Funnel. A reverse proxy on a public host works too, see below.

• Optional: an ntfy topic or a Pushover account for notifications; an OIDC provider such as Keycloak if you want login instead of a static token.

Installation

git clone https://github.com/lorenzkromer/herdr-voice-mcp.git
cd herdr-voice-mcp
npm install
npm run build
npm test

Create the config:

mkdir -p ~/.config/agency
cp config.example.json ~/.config/agency/config.json
chmod 600 ~/.config/agency/config.json
scripts/gen-token.sh

Put the generated token into auth.token and list your projects under projects. Only agents working inside those projects are visible.

Using it locally with Claude Code

No network, no token. Claude Code starts the server itself:

claude mcp add --transport stdio agency -- node /path/to/herdr-voice-mcp/dist/server.js --stdio

Then ask Claude Code: "Let's do a stand-up."

Running it permanently

On macOS, install the server and the notifier as launch agents. They start at login and restart after a crash:

scripts/install-launchd.sh install
scripts/install-launchd.sh status

Logs go to ~/Library/Logs/herdr-voice-server.log and ~/Library/Logs/herdr-voice-notify.log. After a config change run scripts/install-launchd.sh restart.

The HTTP server listens on 127.0.0.1:8791 by default. Test it with:

node scripts/call-tool.mjs list
node scripts/call-tool.mjs status '{"only":"attention"}'

Reaching it from your phone

Option A: Tailscale Funnel with a token (tested)

1. Install and log in to Tailscale on the machine.

2. In the Tailscale admin console, enable HTTPS certificates under DNS.

3. Grant the funnel attribute in the tailnet policy file:

   "nodeAttrs": [
     { "target": ["autogroup:member"], "attr": ["funnel"] }
   ]

4. Publish the port:

   scripts/funnel.sh on

   Tailscale prints the public address, for example https://my-mac.tailnet-name.ts.net/.

5. In the Claude app, add a custom connector. The Claude connector dialog has no field for headers, so the token goes into the URL:

   https://my-mac.tailnet-name.ts.net/mcp/<your token>

   Leave the OAuth fields empty.

6. Enable the connector in a chat and ask "What's new?".

Treat that URL like a password. Anyone who has it can control your agents. To rotate it, generate a new token, update the config, restart, and update the connector.

Option B: reverse proxy on a public host (prepared, not yet verified)

If you already run a reverse proxy with TLS on a public server, it can forward to your machine through a VPN such as WireGuard. The server can listen on several addresses at once, for example localhost for Funnel and the VPN address for the proxy:

"http": { "host": ["127.0.0.1", "10.0.0.5"], "port": 8791 }

A Traefik file-provider example:

http:
  routers:
    agency:
      rule: "Host(`agency.example.com`)"
      entryPoints: [websecure]
      service: agency
      tls: { certResolver: letsencrypt }
  services:
    agency:
      loadBalancer:
        servers:
          - url: "http://10.0.0.5:8791"
        passHostHeader: true

The machine must stay connected to the VPN, including after sleep and network changes. If you add an uptime probe, it must expect 401, never 200.

Authentication

auth.mode selects what the server accepts:

With OAuth configured and public_url set, the server publishes RFC 9728 protected-resource metadata at /.well-known/oauth-protected-resource. It does that only for requests to the public_url host. Every other host, such as the Funnel address, looks like a plain token endpoint. Otherwise the Claude connector would try an OAuth flow there and fail. All other /.well-known/* paths return 404 on purpose, for the same reason.

See config.oauth.example.json. A Keycloak client for this needs:

• confidential client, standard flow only, PKCE S256

• redirect URIs https://claude.ai/api/mcp/auth_callback and https://claude.com/api/mcp/auth_callback

• a group membership mapper (groups claim, no full path) and an audience mapper that puts the client ID into aud of the access token

• a group with exactly the users who may control the agents

In the Claude app, enter the public URL (https://agency.example.com/mcp) and the client ID and secret under the connector's advanced settings.

Notifications

Set notify.enabled to true and pick a provider:

• ntfy: install the ntfy app, subscribe to a long random topic, and put the topic into notify.ntfy.topic. Self-hosted ntfy servers work via notify.ntfy.url and an optional token.

• Pushover: put your user key and application token into notify.pushover.

• log: only writes to the log. Handy for testing without a phone.

Send a test message:

node dist/notify.js --test

On iPhone, enable "Announce Notifications" for the app to have them read into your earbuds.

The notifier subscribes to Herdr's per-pane pane.agent_status_changed events. It only reports agents in allowed projects, and it debounces repeated transitions of the same pane.

Security model

This service forwards instructions to agents that can write to your repositories. It is built for one person and one machine.

• Allow list. Only agents whose workspace repository or working directory belongs to a configured project are visible or reachable. Without projects nothing is visible.

• No shell passthrough. spawn starts only the kinds listed in agent_kinds, only in a project root. keys accepts a fixed small set of keys.

• Authentication on every request. There is no unauthenticated path, not even for status.

• Audit log. Every call is appended to ~/.config/agency/audit.jsonl with time, source, tool, shortened arguments and outcome. Every HTTP request is logged with the token masked.

• Rate limit. 120 requests per minute by default.

• Kill switch. scripts/switch.sh off makes the server answer 503 to everything without touching Herdr. scripts/switch.sh on reverts it.

• Duplicate guard. A dropped connection can make a delivered prompt look failed. send refuses the same text to the same agent for 15 minutes unless force is set.

Keep in mind that everything you dictate passes through the speech recognition of the Claude app and through Claude. Decide consciously whether that is acceptable for your code and your clients.

Configuration reference

The config is read from --config <path>, then $AGENCY_CONFIG, then ~/.config/agency/config.json, then ./config.json. The token can also come from $AGENCY_TOKEN.

Known limitations

• Tool calls must stay short. The Claude connector drops calls that stay open for roughly 30 seconds or more. This is why send never waits for the agent to finish and wait is capped at 25 seconds. A dropped call may surface as a misleading network error in Claude.

• Token in the URL. The Claude connector dialog cannot send headers, so Option A puts the token into the path. It is masked in the server log, but it is visible in the connector settings.

• OAuth with the Claude app is unverified. The server side is implemented and checked against a real Keycloak (valid discovery, rejection of forged tokens). A full login from the Claude app has not been tested yet. It is open whether the app accepts a pre-registered client or insists on dynamic client registration.

• Reverse proxy route is unverified. Option B is documented but has not run end to end.

• Herdr quirks. Herdr's generic pane.updated event does not fire for every status change, for example working to done. The tracker therefore subscribes per pane and resubscribes when agents come and go. Agents that render on the terminal's alternate screen may lose scrollback, so read can only see the current screen for them.

• Durations are approximate. Herdr reports no timestamp for state changes. "For at least N minutes" means the server first saw that state N minutes ago.

• Single user. The stand-up memory and the duplicate guard live in the process and are shared by all callers.

Open points

• End-to-end OAuth login from the Claude app, and a decision on dynamic client registration.

• Verifying the reverse proxy and VPN route, including reconnect after sleep.

• A persistent log of dictated tasks for review at the desk.

• Whether the server should offer git operations or leave them to the agents.

• Better handling of speech recognition errors in file names and technical terms beyond project aliases.

Development

npm run dev                 # HTTP server via tsx
npm run dev -- --stdio      # stdio
npm run dev:notify          # notifier
npm test                    # unit tests with a fake Herdr socket
npm run typecheck

Source layout:

Herdr's wire format, verified against Herdr 0.9.0: newline-delimited JSON over a Unix socket, {"id", "method", "params"} requests, one request per connection. events.subscribe keeps the connection open and streams events. herdr api schema --json prints the full schema.

License

MIT, see LICENSE.