broca-machina

Give any text "brain" a voice — and ears — inside a Discord voice channel.

broca-machina joins a Discord voice channel, transcribes what you say with Whisper, hands the text to whatever produces replies (an LLM, an agent, a shell command, a CLI, or an MCP host), and speaks the answer back into the channel. The "brain" is completely behind a small transport interface, so the same bridge works for a one-line echo toy or a full agent runtime.

It runs under Discord's now-mandatory DAVE end-to-end encryption, which most voice libraries still can't negotiate — see Troubleshooting: why bun below.

flowchart LR
    mic([you speak]) -->|opus 48k to 16k mono| vad
    subgraph loop["broca-machina  ·  voice_loop.js  (bun)"]
      direction LR
      vad["VAD<br/>Silero endpointing<br/>(opt-in)"] --> stt["STT<br/>faster-whisper"]
      clean["cleanForTTS<br/>strip md / emoji / links"] --> tts["TTS<br/>Piper"]
    end
    stt -->|transcript| brain{{"your brain<br/>LLM / agent / CLI<br/>via file / command / mcp"}}
    brain -->|reply text| clean
    tts -->|wav| spk([you hear])

• Speech in: received opus is decoded, downsampled to 16 kHz mono WAV via ffmpeg, and passed to your STT command (default: faster-whisper, CPU-friendly).

• Text out / in: transcripts reach your brain — and replies come back — through a pluggable transport (file, command, or mcp).

• Speech out: reply text is sanitized for speech (markdown, code fences, emoji, and links stripped) and rendered by your TTS command (default: Piper), then played into the channel.

Nothing in the core is app-specific: there are no hard-coded IDs or paths. Everything is driven by one JSON config file.

Quickstart

bun install                              # NOT npm — see "Why bun" below
cp config.example.json config.json       # then edit discord IDs / allowedUserId / stt.cmd / tts.cmd / transport
export DISCORD_VOICE_BOT_TOKEN=...        # your voice bot's token

bun src/voice_loop.js config.json        # run in the foreground
# — or, to background it with a boot/health check and log file:
scripts/voice-up.sh config.json          # scripts/voice-down.sh to stop

You'll also need ffmpeg on PATH and a Python environment with faster-whisper (STT) and piper-tts (TTS). The full walkthrough — creating the Discord bot, intents, invite URL, Python env, and config — is in SETUP.md.

When it's live you'll see:

12:00:01 [loop] logged in as YourBot#1234
12:00:02 [loop] joined voice channel <your-channel-id> (startup)
12:00:02 [loop] LIVE

Related MCP server: Discord MCP Server

Requirements

Security

broca-machina is a pipe from "any sound in a voice channel" to your brain — treat that seriously before you run it in a shared server:

• Fail-closed speaker access. The loop refuses to start unless you set discord.allowedUserId (only that user's speech is transcribed) or explicitly opt into open access with discord.allowAnySpeaker: true. Keep allowedUserId set for any real brain.

• Accepted speech runs with your brain's privileges. With a command or mcp brain — above all a tool-enabled agent like claude -p — a spoken utterance is an untrusted prompt. Anyone you let speak can attempt prompt injection ("ignore your instructions and run…"). Only open the mic (allowAnySpeaker) for a harmless brain such as the echo demo, and treat every transcript as untrusted input to your agent.

• The token never lives in config. It's read from the env var named by discord.tokenEnv (default DISCORD_VOICE_BOT_TOKEN) or from an out-of-repo discord.tokenFile; config.json is gitignored.

• Local IPC only. The warm STT/TTS/VAD servers talk over Unix sockets in .voice-tmp/ — nothing binds a network port. broca-machina assumes a single-user host.

• Use headphones. With an open mic and speakers the bot can hear its own TTS and false-trigger barge-in; headphones (or bargeIn: false) avoid the echo.

See SECURITY.md to report a vulnerability.

Config reference

Configure everything through one JSON file (path from $VOICE_CONFIG or the first CLI argument). Start from config.example.json. Every field the loop reads is listed below; fields marked optional have working defaults and can be omitted.

Presence-gating adds autoJoin, idleLeaveMs, presenceFile, onPresenceEnter, and onPresenceLeave — see Presence-gated auto-join.

STT / TTS engine knobs

The bundled engines read their own environment variables (set them via stt.env / tts.env):

• src/stt.py (faster-whisper): WHISPER_MODEL (default small.en), WHISPER_DEVICE (default cpu), WHISPER_COMPUTE (default int8). WHISPER_MODEL is the STT speed/accuracy lever — base.en transcribes ~2.8× faster than small.en with near-identical output on short utterances; tiny.en is faster still and rougher. Default stays small.en. The warm stt_server.py reads it too, so switch it in stt.env and restart the warm servers.

• src/tts.py (Piper): PIPER_VOICE (default en_US-amy-medium), PIPER_VOICE_DIR (default ~/.cache/broca-machina/piper), VOICE_TTS_SPEED (default 1.0). The voice model is auto-downloaded on first use.

Latency levers

Three opt-in, default-off knobs cut round-trip latency with no hardware change (full analysis in docs/LATENCY.md):

1. WHISPER_MODEL=base.en (in stt.env) — ~2.8× faster STT, default unchanged.

2. vad.enabled: true — Silero-VAD endpointing ends an utterance as soon as you stop talking instead of waiting the fixed endSilenceMs. Reuses the Silero model vendored by faster-whisper (no new dependency; onnxruntime only). Requires the warm VAD server: scripts/warm-servers.sh start vad (or set VOICE_WARM_VAD=1 so plain start includes it). If the server is down, endpointing safely falls back to endSilenceMs. Thresholds are config knobs and benefit from a live tune with a real speaker.

3. ttsPipeline.enabled: true — synthesize + play long replies sentence-by-sentence so time-to-first-audio is one sentence, not the whole reply. Preserves ordering and barge-in.

Perceived latency (distinct from the three above): ackAfterMs: 600 speaks a short filler ~0.6 s after you stop talking when the reply is still being computed — so a slow brain acknowledges you almost immediately while STT + the model run in the background. The default ackPhrase set rotates a few phrases ("One moment." / "Hmm, let me think." / …) so it doesn't chant the same line every turn; all are pre-rendered at startup, a fast reply preempts them, and it's off by default (best for LLM/agent brains; leave off for a fast echo/command brain).

Transports

A transport is the seam between the voice bridge and your brain. Pick one with transport.type.

file (default)

Transcripts are written to transcriptDir/<timestamp>.txt. Replies are read from replyFile — the loop polls that path, and whenever it appears, speaks its contents and deletes it. Whatever writes replyFile becomes the voice. This is the loosest coupling: your brain can be any process, in any language, that watches a directory and drops a text file. See examples/file-transport/ for a reference host loop.

command

Each transcript is handed to transport.cmd as the environment variable $VOICE_TRANSCRIPT. The command's stdout is spoken as the reply. Best for a stateless "one utterance in, one reply out" brain:

"transport": {
  "type": "command",
  "cmd": ["bash", "-lc", "my-llm-cli --prompt \"$VOICE_TRANSCRIPT\""]
}

mcp

A Model Context Protocol transport where the voice loop itself becomes an MCP server. A channel-aware host (e.g. Claude Code) launches broca-machina via .mcp.json; spoken turns arrive at the host as a notifications/claude/channel event, and the host replies by calling the speak tool, which voices the text back into the channel.

"transport": { "type": "mcp", "source": "voice", "deliver": "channel" }

Register it with your host: copy .mcp.json.example to .mcp.json, set an absolute cwd, and point it at adapters/mcp.config.example.json (edit the Discord IDs + stt/tts first):

{
  "mcpServers": {
    "broca-machina": {
      "command": "bun",
      "args": ["src/voice_loop.js", "adapters/mcp.config.example.json"],
      "cwd": "/abs/path/to/broca-machina"
    }
  }
}

The speak tool is the only capability exposed to the host — it voices text and nothing else (no file or command access). Inbound speech uses an experimental claude/channel notification, so the MCP path is first-class with Claude Code / channel-aware hosts today; a host that can't receive channel events still gets speak. The transport.deliver switch is the extension seam for future generic-host inbound modes — see docs/ARCHITECTURE.md.

Warm model servers (lower latency)

stt.py / tts.py load their models on every call (~1.1 s each). To skip that cost, run persistent servers that keep faster-whisper and Piper warm, and point the config at the drop-in clients:

VOICE_PY=/path/to/python \
PIPER_VOICE_DIR=/path/to/piper WHISPER_MODEL=base.en \
  scripts/warm-servers.sh start        # start | stop | status | restart

Then in config.json, swap the commands (identical CLI, so it's a 1-line change each):

"stt": { "cmd": ["/path/to/python", "src/stt_client.py"] },
"tts": { "cmd": ["/path/to/python", "src/tts_client.py"] }

The clients talk to the servers over Unix sockets in .voice-tmp/ and fall back to a cold in-process load if a server is down — pure upside, never a new failure mode. Measured ~2.2× faster STT+TTS per exchange (≈4× with WHISPER_MODEL=base.en). See docs/LATENCY.md for the cold-vs-warm numbers and the ranked list of further no-hardware wins.

Presence-gated auto-join

By default the bot joins the voice channel at startup and stays there for the whole session. A long-lived idle voice connection is fine functionally, but it silently rots — Discord drops inactive voice sessions and net blips go unnoticed until you try to talk — and it holds the warm-server RAM even when nobody's around.

Set autoJoin: true for a presence-gated lifecycle instead (presence is keyed to allowedUserId; with allowAnySpeaker: true it tracks any non-bot member of the channel):

• The gateway stays connected as a cheap presence listener — no voice UDP, no models loaded.

• When the allowed user joins the voice channel, the bot fires onPresenceEnter (detached) and joins. The hook is meant for warming models: because it runs concurrently with the multi-second voice handshake and the human gap before the first word, warm-up is effectively free, so the first utterance lands on warm servers.

• When the user leaves, an idle timer starts. If they haven't returned within idleLeaveMs (default 600000 = 10 min), the bot leaves the channel and fires onPresenceLeave (e.g. spin the warm servers down to reclaim RAM). A quick rejoin cancels the pending leave, so brief drops don't thrash the connection.

• presenceFile (optional, any mode): a marker file kept in existence exactly while the user is in the voice channel — lets external tooling gate on "is the human here" without polling Discord. Removed on graceful shutdown.

"allowedUserId": "YOUR_DISCORD_USER_ID",
"autoJoin": true,
"idleLeaveMs": 600000,
"onPresenceEnter": { "cmd": ["bash", "scripts/warm-servers.sh", "start"], "env": {} },
"onPresenceLeave": { "cmd": ["bash", "scripts/warm-servers.sh", "stop"],  "env": {} }

If the user speaks in the ~2.5 s before the warm STT server is up, the drop-in client falls back to a cold in-process load for that one utterance — it's slower but never dropped, so no first-word queue is needed. Leave autoJoin unset (false) to keep the always-in-channel behavior.

Adapters

adapters/ holds ready-to-copy config templates for specific host wirings. adapters/mcp.config.example.json is a complete MCP-transport setup (the voice loop as an MCP server): copy it, set your Discord IDs, point stt/tts at your Python, and register it with your host via .mcp.json.example. Nothing in src/ is host-specific — every wiring is just a JSON config.

Troubleshooting

The bot connects but immediately drops / close code 4017 / "never Ready". This is the DAVE end-to-end-encryption handshake failing — almost always because the wrong @discordjs/voice shipped. Confirm you installed with bun (bun install) and that node_modules/@discordjs/voice/package.json reads 0.19.2, not 0.18.x. npm install on Node 20 silently resolves to the pre-DAVE 0.18.0, which cannot complete the handshake on today's Discord.

Why bun, and why the exact versions? Discord made DAVE E2EE mandatory for voice in early 2026. Support landed in @discordjs/voice 0.19, which bundles @snazzah/davey and requires Node ≥ 22.12. bun is a modern runtime that satisfies that requirement out of the box, so bun install + bun src/voice_loop.js is the supported path. Running under an older Node lets the resolver fall back to 0.18.0 (no DAVE) and the join fails. See docs/ARCHITECTURE.md for the full rationale.

It logs in but "no bot token in $DISCORD_VOICE_BOT_TOKEN" / login FAILED. Export the token into the env var named by discord.tokenEnv (default DISCORD_VOICE_BOT_TOKEN) before launching, or point voice-up.sh at a .env with VOICE_TOKEN_ENVFILE=/path/to/.env.

It refuses to start: "no discord.allowedUserId set". That's the fail-closed guard (see Security). Set discord.allowedUserId to your user ID, or discord.allowAnySpeaker: true to explicitly accept anyone.

I already run a Discord bot and broca-machina won't stay connected. Discord allows one gateway connection per bot. If your existing text bot is already connected on that application's token, create a second Discord bot (its own application + token) for voice. Two bots, two gateways.

No audio is transcribed. Check that (a) the bot actually joined the channel (the [loop] joined line — with autoJoin, that only appears once you're in the voice channel), (b) if allowedUserId is set, that you are that user, and (c) you're speaking longer than minUtteranceSec. Very short or silent-ish utterances and the sttNoiseDrop list are filtered out by design — the log prints [recv] … too short or [stt] drop: "…" when that happens.

ffmpeg not found. ffmpeg is probed at startup — if it's missing the loop exits immediately with a clear message rather than failing per-utterance later. Install it (apt install ffmpeg / brew install ffmpeg) and restart.

It's slow on CPU. The defaults are CPU-only. src/stt.py reloads the Whisper model on every utterance (a deliberate simplicity trade-off noted in the file), so expect a few seconds of round-trip latency. For lower latency: use a smaller/faster Whisper model (WHISPER_MODEL=tiny.en), run STT on a GPU (WHISPER_DEVICE=cuda), replace stt.cmd with a persistent transcription server, or enable the warm servers.

Project layout

src/voice_loop.js       the bridge (bun): receive → STT → transport → TTS → play
src/stt.py              default STT (faster-whisper), per-call model load
src/tts.py              default TTS (Piper), per-call load, pitch-preserved speed control
src/stt_server.py       warm STT server: loads the model once, serves over a Unix socket
src/tts_server.py       warm TTS server: loads Piper once, serves over a Unix socket
src/stt_client.py       drop-in for stt.py; talks to the warm server, cold fallback if down
src/tts_client.py       drop-in for tts.py; talks to the warm server, cold fallback if down
src/vad.py              Silero-VAD endpointer (opt-in early end-of-utterance)
src/vad_server.py       warm VAD server: streams PCM in, reports end-of-speech
src/vad_stream.js       loop-side VAD client (Unix-socket framing)
src/_voicesock.py       shared length-prefixed-JSON socket framing for server/client
scripts/warm-servers.sh start/stop/status the warm STT+TTS(+VAD) servers
scripts/voice-up.sh     background-start with a boot/health check + log
scripts/voice-down.sh   stop the backgrounded loop
config.example.json     copy to config.json and edit
requirements.txt        python deps for the bundled engines (faster-whisper, piper, …)
adapters/               ready-to-copy config templates (e.g. mcp.config.example.json)
examples/               runnable brains: echo, ollama, claude-cli, file-transport
test/                   selftests (lifecycle, TTS pipeline + ack, VAD) — CI runs these
.mcp.json.example       example MCP registration for a channel-aware host (Claude Code)
assets/                 logo + icon (SVG)
docs/ARCHITECTURE.md    pipeline, transport abstraction, DAVE/bun rationale
SETUP.md                end-to-end setup walkthrough
CONTRIBUTING.md         dev setup, adding transports/engines, PR expectations

License

MIT — see LICENSE. © 2026 Cody Slater.