# claude-voice-bridge

Talk to Claude Code, hands-free. A ~450-line bridge that exposes headless
Claude Code (`claude -p`) as a remote MCP server, plus a one-tap browser voice
client built on OpenAI's Realtime API (`gpt-realtime-2.1` — the same voice
tech as ChatGPT).

```
iPhone/Mac Safari ──WebRTC audio──▶ OpenAI gpt-realtime-2.1
                                        │  (MCP tool calls over HTTPS)
                                        ▼
              Tailscale Funnel ▶ this bridge (127.0.0.1:8765)
                                        │  spawns
                                        ▼
                    claude -p <task> --output-format stream-json
                         in an allow-listed project directory
```

## What's in this repo

- `server.py` — the bridge itself: MCP server + OpenAI Realtime secret
  minting + task/tmux orchestration
- `static/` — the voice client page and a live-status watch page
- `playground/` — a tiny example project the bridge is allowed to touch,
  plus a demo transcript panel
- `tests/`, `test_client.py` — an end-to-end MCP smoke test and a fake
  `claude` CLI stand-in for testing without real credentials/API calls

**Not in this repo** (external dependencies you install/configure yourself):
the [Claude Code CLI](https://docs.claude.com/en/docs/claude-code) itself,
`tmux` (used for native/interactive mode — see below), the
[Claude in Chrome](https://claude.ai) browser extension (an independent tool
some Claude Code setups use for browser automation; this bridge doesn't
orchestrate it directly, Claude Code does), and Tailscale. This repo is the
glue between voice input and a Claude Code session — not a bundle of those
tools.

**Why not ChatGPT's own voice mode?** Verified 2026-07-17: ChatGPT voice
(GPT-Live) cannot invoke connectors — built-in or custom MCP — and custom-GPT
Actions are also skipped in voice. The Realtime API is OpenAI's supported way
to get the same voice models *with* tool calling. (The bridge still works as a
ChatGPT **text-chat** connector; see below.)

## Requirements

- macOS with the [Claude Code CLI](https://docs.claude.com/en/docs/claude-code)
  installed and authenticated (`claude` on PATH or a known path)
- Python 3.11+
- An [OpenAI](https://platform.openai.com/) account with Realtime API access
- [Tailscale](https://tailscale.com/) with Funnel enabled, to reach the
  bridge from your phone off your home network (optional if you only use it
  on localhost)
- `tmux`, if you want native/interactive mode (see below) — not required for
  headless mode

## Setup

```bash
git clone https://github.com/<you>/claude-voice-bridge.git
cd claude-voice-bridge
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

cp config.example.json config.json
chmod 600 config.json
# edit config.json:
#   - "token": generate a long random string, e.g. `openssl rand -hex 16`
#   - "openai_api_key": your OpenAI key
#   - "projects": name -> absolute path allowlist for what Claude Code may touch
#   - "claude_bin": absolute path to your claude binary
```

1. **Re-auth the CLI (once).** Either:
   - run `claude /login` in a terminal, **or**
   - run `claude setup-token` and paste the long-lived token into
     `config.json` as `"claude_oauth_token": "..."` (best for unattended use —
     survives future keychain drift).
2. **(Optional) Expose it off-network:** `tailscale funnel --set-path
   /<your-token> 8765`, then set `public_base` in `config.json` to the funnel
   hostname it prints.
3. **Start the bridge:** `./run.sh` (keep the terminal tab open, or see
   "Run at login" below).
4. **Open the voice page** (Safari on iPhone or Mac):
   the `voice page` URL printed at server start — tap **Connect**, allow the
   mic, talk.

The Tailscale Funnel route persists across reboots; only `run.sh` needs
restarting.

## Talking to it

- "What projects can you work on?"
- "In the playground project, build a snake game in a single HTML file."
- "How's it going?" / "What's the status?"
- "What did it change?" (final outcome)
- "Tell it to also add a high-score display." (resumes the same session)
- "Stop the task."
- "Use opus for this one" / "let it run shell commands" (maps to
  `model` / `allow_bash`).

Tasks run in the background for minutes; start one, chat or hang up, ask for
updates whenever. Claude Code ends every task with a `SPOKEN:` summary written
to be read aloud.

## Endpoints (all under the secret path token from `config.json`)

| path | what |
|---|---|
| `/<token>/voice` | voice client page |
| `/<token>/rt-secret` | POST → mints an ephemeral OpenAI Realtime secret (session config incl. MCP lives server-side here) |
| `/<token>/mcp` | the MCP endpoint (streamable HTTP, stateless JSON) |
| `/<token>/health` | liveness + task states |

## Native mode (default)

With `"mode": "native"` in config, voice requests don't run headless — they
land in a REAL interactive Claude Code session running in tmux, surfaced in a
Terminal window on the Mac (`tmux attach -t cv-<project>` from anywhere too).
Three-way collaboration: you watch the native TUI and can type into it
directly (Claude Code queues input mid-run), the voice agent injects prompts
via tmux and reads replies from the live session transcript, and each project
is one continuous session with full context. "Stop" from voice sends an
interrupt (Escape) to the window. `background: true` requests still run
headless and invisible. Set `"mode": "headless"` to revert entirely.

## MCP tools

`claude_code(request, project?, model?, effort?, background?)` — the front
door. ANY request goes through it: questions, inspections, debugging, code
changes. It waits up to `wait_s` (90s): quick things return the answer
directly in `say`; longer work converts to a background task (state
`running`). `background: true` skips the wait. `project` optional — omitted
requests root at `home` (your home directory) and Claude Code roams from
there.

Supporting tools: `list_projects` · `claude_status(task_id?)` ·
`claude_result(task_id?)` · `claude_reply(message, task_id?)` (resumes the
same session with full context, same wait-or-background behavior) ·
`claude_stop(task_id?)`.

All tools return a `say` field written for speech plus structured detail;
`task_id` defaults to the most recent task. Finished tasks carry
`session_id` and a `resume_command` (`cd <project> && claude --resume <id>`)
— voice sessions are ordinary Claude Code sessions in `~/.claude/projects/`,
so you can adopt any of them interactively at your desk.

## ChatGPT text-chat connector (optional second consumer)

ChatGPT (web, Plus/Pro) → Settings → Apps & Connectors → Advanced settings →
enable **Developer mode** → create connector with the `/<token>/mcp` URL, no
auth. Works in text conversations (including keyboard dictation); write tools
require a per-conversation confirmation tap. Voice mode will ignore it — see
above.

## config.json

| key | meaning |
|---|---|
| `token` | secret path segment gating everything (rotate: change it, re-run the funnel command with the new path, remove the old: `tailscale funnel --set-path /<old> off`) |
| `projects` | name → directory allowlist; Claude Code only ever runs inside these |
| `default_model` | `sonnet` (voice can say "use opus/haiku" per task) |
| `permission_mode` | `acceptEdits` — file edits auto-approved inside the project; shell commands are NOT unless `allow_bash`/`allowed_tools` |
| `allowed_tools` | extra always-allowed tools, e.g. `["Bash"]` for trusted setups |
| `claude_oauth_token` | optional long-lived headless token from `claude setup-token` |
| `openai_api_key` | used only to mint ephemeral Realtime secrets |
| `realtime_model` | `gpt-realtime-2.1` (flagship; `gpt-realtime-2.1-mini` is ~3× cheaper, slightly weaker tool use) |
| `claude_bin` | absolute path to the claude binary (`~/.claude/local/claude` — it's a zsh alias, not on PATH) |

Realtime voice cost ballpark: ~$0.04–0.10/min flagship, ~1/3 on mini —
separate from Claude usage.

## Security model — read this before exposing it

- **If you funnel this to the public internet, the token path is the entire
  gate on code execution.** There's no additional login, MFA, or IP allowlist
  by default — anyone who obtains the URL can drive Claude Code inside your
  allow-listed project directories. Treat the URL exactly like a password:
  don't paste it into chat logs, screenshots, or public issues. `config.json`
  should stay `chmod 600` and must never be committed (see `.gitignore`).
- **`permission_mode` in `config.json` controls the blast radius.** The
  shipped example defaults to `acceptEdits` (file edits inside the
  allow-listed project auto-approve; shell commands do **not**, unless you
  add `Bash` to `allowed_tools`). Setting `permission_mode` to
  `bypassPermissions` removes all permission checks — voice tasks then get an
  unconfirmed shell, and the token path becomes the *only* thing standing
  between the public internet and full code execution on your machine. Only
  do this if you understand and accept that tradeoff.
- **`projects` is an allowlist, not a sandbox.** Claude Code can read/write
  anywhere inside a listed directory (and, if shell access is enabled, run
  arbitrary commands as your user). Don't point it at anything you wouldn't
  hand a well-meaning but unsupervised script.
- **Rotate the token** if you ever suspect it leaked: change `token` in
  `config.json`, re-run the funnel command with the new path, then remove the
  old route (`tailscale funnel --set-path /<old-token> off`).
- **Kill switches:** `tailscale funnel --set-path /<token> off` (removes just
  this route), or stop the server (`Ctrl-C` on `run.sh`, or kill the
  process).
- **Secrets that must never be committed:** `token`, `openai_api_key`,
  `claude_oauth_token` — all live only in your local `config.json`, which is
  gitignored. Use `config.example.json` as the template.

## Run at login (optional)

launchd + zsh can't read `~/Desktop` (TCC) — use the compiled ad-hoc-signed
runner-binary trick already validated on the PokerAdvisor auto-trigger, with
`run.sh` as the target. Or just keep a terminal tab.

## Troubleshooting

- **"Failed to authenticate: OAuth session expired"** in task results → step 1
  above.
- **rt-secret 4xx** → OpenAI key invalid, or `realtime_model` name has
  drifted; check the error JSON the endpoint returns.
- **Voice page won't connect on iPhone** → must be real Safari (not an
  in-app webview), and audio may route to the speaker instead of AirPods —
  known iOS WebRTC quirk.
- **Transcript only shows the assistant** → by design (input transcription is
  off to keep the session config minimal); add `audio.input.transcription`
  in `_mint_secret` if wanted.
- Per-task raw stream-json logs live in `tasks/task-N.jsonl`.