# Voice MCP Server

MCP stdio server that lets Codex speak on demand. Codex starts the local stdio
process, calls `voice_speak`, the server generates the audio, plays it through
the system output, returns the tool result, and then waits for the next MCP
request from the client.

## Quick Start

```bash
npm install
npm run build
```

Set a real provider before starting the MCP server. To use ElevenLabs:

```bash
export VOICE_PROVIDER="elevenlabs"
export ELEVENLABS_API_KEY="..."
npm run build
```

To use OpenAI Realtime streaming audio:

```bash
export VOICE_PROVIDER="openai-realtime"
export OPENAI_API_KEY="..."
export OPENAI_REALTIME_MODEL="gpt-realtime-2.1-mini"
npm run build
```

The Realtime provider is text-in/audio-out. It does not listen to the
microphone. `voice_speak` sends the requested text as a Realtime conversation
item, receives `response.output_audio.delta` chunks, and streams 24 kHz mono
PCM audio to the local player as it arrives.

The transport is MCP over `stdio`; Codex starts `dist/index.js` directly.
You do not run this as a daemon or keep a separate terminal open.
After changing configuration or rebuilding, restart the MCP client/session so it
loads the updated environment and compiled code.

## Provider Setup

The supported speech providers are `elevenlabs` and `openai-realtime`. The
server does not silently fall back to a local test tone when credentials are
missing; it fails startup with a configuration error instead.

| Provider | Use it when | Billing model | Required key |
| --- | --- | --- | --- |
| `elevenlabs` | You want regular text-to-speech with ElevenLabs voices, free-tier testing, and cached generated audio. | ElevenLabs credits/characters. The free plan is assigned automatically when you sign up. | `ELEVENLABS_API_KEY` |
| `openai-realtime` | You want OpenAI Realtime voices and low-latency streaming from the Realtime API. | OpenAI API usage. This consumes API credits and is separate from any ChatGPT/Codex subscription. | `OPENAI_API_KEY` |

### ElevenLabs Free API Key

1. Create a free ElevenLabs account at [elevenlabs.io](https://elevenlabs.io/).
2. Open the ElevenLabs dashboard.
3. Go to `Developers` in the left sidebar, then `API Keys`.
4. Create a new API key. For this server, enable Text to Speech access; setting
   a key-specific credit limit is a good guardrail.
5. Copy the key when it is created. ElevenLabs only shows the full key once.
6. Set it in your MCP config or environment:

```bash
export VOICE_PROVIDER="elevenlabs"
export ELEVENLABS_API_KEY="..."
export ELEVENLABS_MODEL_ID="eleven_flash_v2_5"
export ELEVENLABS_FORMAT="mp3"
```

The ElevenLabs API authenticates requests with the `xi-api-key` header. Keep the
key secret; do not commit it or expose it in client-side code. See the official
[ElevenLabs API key docs](https://elevenlabs.io/docs/help-center/technical/how-do-i-authorize-myself-using-an-api-key)
and [authentication reference](https://elevenlabs.io/docs/api-reference/authentication).
The free plan includes a monthly credit quota; generated speech consumes those
credits even when you use the API.

### OpenAI Realtime API Key

Use `openai-realtime` only when you intentionally want to use OpenAI API
billing for speech output. This server defaults to `gpt-realtime-2.1-mini` for
OpenAI Realtime because it is much cheaper and is usually enough for text-to-speech
playback. Switch to `gpt-realtime-2.1` only when you need the higher-quality model.

```bash
export VOICE_PROVIDER="openai-realtime"
export OPENAI_API_KEY="..."
export OPENAI_REALTIME_MODEL="gpt-realtime-2.1-mini"
export OPENAI_REALTIME_VOICE="marin"
export OPENAI_REALTIME_FORMAT="pcm16"
```

This provider sends text to the Realtime API and receives streamed 24 kHz mono
PCM audio. It does not listen to the microphone. Realtime is billed by tokens
used, not by wall-clock session time alone. An idle open session should not be
the expensive part; generated audio output tokens are. Apps that continuously
send microphone audio can still spend audio input tokens while they are open,
but this MCP does not send microphone audio. For cost predictability, this
server closes the Realtime connection after each generated response instead of
keeping a long conversation session open.

### Pricing Reference

Prices change, so check the provider pricing pages before heavy usage:
[ElevenLabs API pricing](https://elevenlabs.io/pricing/api?price.platform=api)
and [OpenAI pricing](https://developers.openai.com/api/docs/pricing). The
figures below were checked on 2026-08-02.

| Provider/model | Unit | Price | Notes |
| --- | --- | --- | --- |
| ElevenLabs Flash/Turbo TTS | 1,000 text characters | $0.05 | The default `eleven_flash_v2_5` falls in this low-latency family. |
| ElevenLabs Multilingual v2/v3 TTS | 1,000 text characters | $0.10 | Relevant if you switch to a multilingual model. |
| OpenAI `gpt-realtime-2.1` text input | 1M text tokens | $4.00 | This server sends text input. |
| OpenAI `gpt-realtime-2.1` text output | 1M text tokens | $24.00 | Usually not the main cost for this server. |
| OpenAI `gpt-realtime-2.1` audio input | 1M audio tokens | $32.00 | Not normally used here because the server does not send microphone audio. |
| OpenAI `gpt-realtime-2.1` audio output | 1M audio tokens | $64.00 | This is the main OpenAI Realtime cost for speech playback. |
| OpenAI `gpt-realtime-2.1-mini` text input | 1M text tokens | $0.60 | Recommended OpenAI Realtime default for this server. |
| OpenAI `gpt-realtime-2.1-mini` text output | 1M text tokens | $2.40 | Usually not the main cost for this server. |
| OpenAI `gpt-realtime-2.1-mini` audio output | 1M audio tokens | $20.00 | Lower-cost Realtime option if quality is good enough for your use case. |

Example phrase:

```text
Hello, this is a quick voice test.
```

That phrase has 34 text characters.

| Provider | Example calculation | Estimated cost |
| --- | --- | --- |
| ElevenLabs Flash/Turbo | `34 / 1000 * $0.05` | `$0.0017` |
| ElevenLabs Multilingual v2/v3 | `34 / 1000 * $0.10` | `$0.0034` |
| OpenAI Realtime text input | Text input is roughly `9 / 1,000,000 * $4`. | `$0.000036` |

For OpenAI Realtime, the expensive line item is usually generated `audio,
output`. There is no fixed character-to-audio conversion, but a practical
budgeting rule is to estimate about 1,200 audio output tokens for each minute of
generated assistant speech. Check the OpenAI dashboard for the actual billed
token usage.

```text
estimated_audio_output_cost =
  generated_speech_minutes * 1,200 * price_per_1m_audio_output_tokens / 1,000,000
```

Estimated OpenAI Realtime audio-output cost:

| Generated speech | `gpt-realtime-2.1` audio output | `gpt-realtime-2.1-mini` audio output |
| --- | --- | --- |
| 1 minute | `1 * 1,200 * $64 / 1M = ~$0.08` | `1 * 1,200 * $20 / 1M = ~$0.02` |
| 10 minutes | `10 * 1,200 * $64 / 1M = ~$0.77` | `10 * 1,200 * $20 / 1M = ~$0.24` |
| 60 minutes | `60 * 1,200 * $64 / 1M = ~$4.61` | `60 * 1,200 * $20 / 1M = ~$1.44` |

The OpenAI examples are illustrative because Realtime billing is based on actual
token usage, including audio output tokens. Generated ElevenLabs audio is cached
by this server by default; OpenAI Realtime streaming is not cached, so replaying
the same text with OpenAI spends API credits again.

Example MCP client configuration:

```json
{
  "mcpServers": {
    "voice": {
      "command": "node",
      "args": ["./dist/index.js"],
      "cwd": "/Users/francescperez/Documents/voice mcp server",
      "env": {
        "VOICE_PROVIDER": "elevenlabs",
        "ELEVENLABS_API_KEY": "...",
        "ELEVENLABS_VOICE_ID": "hpp4J3VqNfWAUOO0d1Us",
        "ELEVENLABS_MODEL_ID": "eleven_flash_v2_5",
        "ELEVENLABS_FORMAT": "mp3"
      }
    }
  }
}
```

Example OpenAI Realtime MCP client configuration:

```json
{
  "mcpServers": {
    "voice": {
      "command": "node",
      "args": ["./dist/index.js"],
      "cwd": "/Users/francescperez/Documents/voice mcp server",
      "env": {
        "VOICE_PROVIDER": "openai-realtime",
        "OPENAI_API_KEY": "...",
        "OPENAI_REALTIME_MODEL": "gpt-realtime-2.1-mini",
        "OPENAI_REALTIME_VOICE": "marin",
        "OPENAI_REALTIME_FORMAT": "pcm16"
      }
    }
  }
}
```

The same block is available in `.mcp.example.json`. Keep real secrets in your
local `.env` or `.mcp.json`; both are ignored by git.

## Tools

- `voice_speak`: generate and play speech.
- `voice_wait`: wait for a playback id to finish.
- `voice_stop`: stop current playback and clear pending items.
- `voice_status`: return queue/current/history state plus provider diagnostics.
- `voice_is_speaking`: return whether audio is currently playing or being prepared.

`voice_speak` accepts:

```json
{
  "text": "Hello",
  "waitUntilFinished": true,
  "voiceId": "optional voice id",
  "speed": 1.0,
  "volume": 1.0,
  "policy": "enqueue"
}
```

Queue policies are `enqueue`, `interrupt`, `replace`, and `discard`.

## Resources

- `voice://history`: transcript/playback history as JSON.
- `voice://status`: current playback status as JSON.

## Notifications

The server emits custom JSON-RPC notifications:

- `voice.started`
- `voice.progress`
- `voice.finished`
- `voice.stopped`
- `voice.error`

## Configuration

Configuration can come from environment variables or from a file pointed at by
`VOICE_MCP_CONFIG` or `--config`. JSON is supported, plus a flat `key = value`
or `key: value` file for the keys shown in the design spec.

Example:

```toml
provider = "elevenlabs"
api_key = "${ELEVENLABS_API_KEY}"
voice_id = "hpp4J3VqNfWAUOO0d1Us"
model_id = "eleven_flash_v2_5"
format = "mp3"
volume = 1.0
speed = 1.0
output_device = "default"
cache = true
```

Only `ELEVENLABS_API_KEY` is needed for normal ElevenLabs use. Everything else
has a default:

- `VOICE_PROVIDER`: set `elevenlabs` or `openai-realtime`. If unset, the server
  auto-selects `elevenlabs` only when `ELEVENLABS_API_KEY` exists; otherwise it
  fails startup.
- `ELEVENLABS_VOICE_ID`: defaults to `hpp4J3VqNfWAUOO0d1Us`.
- `ELEVENLABS_MODEL_ID`: defaults to `eleven_flash_v2_5`.
- `ELEVENLABS_MODEL`: legacy alias for `ELEVENLABS_MODEL_ID`.
- `ELEVENLABS_FORMAT`: defaults to `mp3`; `mp3` maps to ElevenLabs
  `mp3_44100_128`. `pcm`, `pcm16`, and `audio/pcm` map to `pcm_24000`
  and are wrapped as WAV before local playback. Other ElevenLabs
  `output_format` values can be passed through directly.
- `OPENAI_API_KEY`: required when `VOICE_PROVIDER=openai-realtime`.
- `OPENAI_REALTIME_VOICE`: defaults to `marin`.
- `OPENAI_REALTIME_MODEL`: defaults to `gpt-realtime-2.1-mini`. Use
  `gpt-realtime-2.1` only when you need the higher-quality model.
- `OPENAI_REALTIME_FORMAT`: defaults to `pcm16`, which maps to 24 kHz mono PCM.
- `OPENAI_REALTIME_INSTRUCTIONS`: optional narrator instructions for the Realtime model.
- `VOICE_VOLUME`: defaults to `1.0`.
- `VOICE_SPEED`: defaults to `1.0`.
- `VOICE_OUTPUT_DEVICE`: defaults to `default`.
- `AUDIO_PLAYER_CMD`: optional player command override.
- `AUDIO_STREAM_PLAYER_CMD`: optional streaming player command override. Placeholders:
  `{sampleRate}`, `{channels}`, `{encoding}`, and `{volume}`.
- `VOICE_MCP_CONFIG`: optional external config file.
- `VOICE_CACHE`: defaults to `true`. Cache keys include the provider and
  output format, so switching between providers or ElevenLabs formats does not
  reuse stale audio from another mode.
- `VOICE_CACHE_DIR`: defaults to `.voice-cache`.

## Audio Playback

On macOS the server uses `afplay`. On Linux it uses `aplay` for WAV and
`ffplay` for other formats. You can override the command with
`AUDIO_PLAYER_CMD`.

ElevenLabs emits MP3 by default; PCM ElevenLabs responses are wrapped as WAV
before playback.
OpenAI Realtime streaming uses `ffplay` by default:

```bash
ffplay -nodisp -autoexit -loglevel quiet -f s16le -sample_rate 24000 -ch_layout mono -volume 100 -i pipe:0
```

## Troubleshooting

If playback sounds like a simple tone or beep, you are probably running an older
build or an already-started MCP process from before this change. Rebuild,
restart the MCP client/session, and call `voice_status` to confirm that
`provider` is `elevenlabs` or `openai-realtime`.