Skip to main content
Glama
ElevenLabs Voice-to-Voice Agent

ElevenLabs Voice-to-Voice Agent

by Begwen

ElevenLabs Voice-to-Voice Agent — MCP Server

A full voice-to-voice AI agent built on the Model Context Protocol. Speak into your microphone, the agent thinks, and ElevenLabs speaks back — all orchestrated through MCP tools, prompts, and resources.

Works with Anthropic Claude or OpenAI GPT — no Claude Desktop required.


MCP Primitives Implemented

Tools (7)

Tool

Description

list_voices

List all available ElevenLabs voices

get_voice

Get detailed info about a specific voice

list_models

List available TTS/STT models

text_to_speech

Convert text → MP3, save to disk

speech_to_text

Transcribe an audio file → text

get_user_info

Account and character usage info

get_history

Retrieve generation history

Prompts (4) — prompt-centric MCP

Prompt

Args

What it does

voice_agent_persona

name, personality, [voice_style]

Creates a named agent persona; picks the best matching voice and speaks every reply aloud

start_voice_session

[topic], [language], [voice_id]

Initialises a live voice loop — agent listens and responds in spoken audio

find_voice_for_role

role

Lists voices, picks top 3 for the role, generates audio samples for each

voice_showcase

phrase, [count]

Renders the same phrase across N diverse voices for side-by-side comparison

Resources (2)

URI

Description

elevenlabs://voices

Live voice list — IDs, categories, labels

elevenlabs://models

Available TTS/STT models and capabilities


Related MCP server: ElevenLabs MCP Server

Prerequisites

  • Node.js ≥ 18

  • sox (for microphone recording in /voice mode)

    brew install sox        # macOS
    sudo apt install sox    # Ubuntu/Debian
  • An ElevenLabs API key

  • An LLM key: Anthropic or OpenAI


Setup

npm install
npm run build

Environment Variables

Variable

Required

Description

ELEVENLABS_API_KEY

Yes

ElevenLabs API key

LLM_PROVIDER

No

"anthropic" (default) or "openai"

ANTHROPIC_API_KEY

If using Anthropic

Claude API key

OPENAI_API_KEY

If using OpenAI

GPT API key

ELEVENLABS_OUTPUT_DIR

No

Where MP3 files are saved (defaults to cwd)

Export before running (the server reads from process.env directly — .env is not auto-loaded):

export ELEVENLABS_API_KEY="your-key"
export LLM_PROVIDER="anthropic"
export ANTHROPIC_API_KEY="your-key"

Usage

Start the client

npm run client
ElevenLabs Voice Agent — MCP Client (provider: anthropic)

Connected! 7 tools: list_voices, get_voice, list_models, text_to_speech, speech_to_text, get_user_info, get_history
Commands: /prompts  /prompt <name> [args]  /voice  exit

Text chat

You: List all available voices
  [tool: list_voices]
Assistant: Here are the available voices...

You: Convert "Welcome to my demo" to speech using a deep male voice
  [tool: list_voices]
  [tool: text_to_speech]
Assistant: Audio saved to: ./output_1234.mp3

List and invoke prompts

You: /prompts

Available prompts:
  voice_agent_persona — Create a voice AI agent with a specific persona...
    args: name, personality, [voice_style]
  start_voice_session — Initialize a live voice conversation...
    args: [topic], [language], [voice_id]
  find_voice_for_role — Analyze and recommend voices for a use case...
    args: role
  voice_showcase — Render a phrase in multiple voices...
    args: phrase, [count]

You: /prompt voice_agent_persona name="Aria" personality="warm and professional"
  [tool: list_voices]
  [tool: text_to_speech]
Assistant: Hi, I'm Aria! How can I help you today?
  [Audio plays automatically]

Voice conversation mode

You: /voice

[Voice Mode] The agent will listen and speak back.
[Voice Mode] Type 'exit' at any prompt to return to text mode.

Press ENTER to speak (or type 'exit'):
  Recording... Press ENTER to stop:
  Transcribing...

You: "What voices does ElevenLabs have?"
  [tool: list_voices]
  [tool: text_to_speech]
Assistant: ElevenLabs has over 30 voices...
  [Audio plays automatically]

Press ENTER to speak (or type 'exit'): exit

[Returned to text mode]

ElevenLabs API Key Permissions

When creating your API key, enable:

Permission

Setting

Text to Speech

Access

Speech to Text

Access

Voices

Read

Models

Access

History

Read

User

Read

Everything else can stay at No Access.


With Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "elevenlabs": {
      "command": "node",
      "args": ["/absolute/path/to/build/index.js"],
      "env": { "ELEVENLABS_API_KEY": "your-key" }
    }
  }
}

With Claude Code

claude mcp add elevenlabs node /absolute/path/to/build/index.js

Architecture

src/
  elevenlabs-client.ts  — ElevenLabs REST API wrapper (TTS + STT + voices + models)
  index.ts              — MCP server: 7 tools, 4 prompts, 2 resources
  client.ts             — CLI client: text REPL + /voice loop + /prompt commands
You (mic/keyboard)
    │
    ▼
client.ts ──────────────────────────────► LLM API (Claude / GPT)
    │  MCP stdio transport                       │
    ▼                                            │ tool calls
index.ts (MCP Server)  ◄────────────────────────┘
    │
    ▼
elevenlabs-client.ts ──► ElevenLabs REST API
                              (TTS + STT + voices + models)

Voice loop flow:

  1. Mic → sox → WAV file

  2. speech_to_text tool → transcript text

  3. LLM (with VOICE_SYSTEM prompt) → text response + text_to_speech call

  4. text_to_speech tool → MP3 file

  5. afplay / mpg123 → audio plays from speakers

  6. Loop back to step 1


License

MIT

Available Tools

7 tools
get_historyB

Retrieve the history of TTS generations from ElevenLabs.

Inputs: page_size.

get_user_infoA

Get the current ElevenLabs user/subscription info including character usage and limits.

Inputs: no named parameters.

get_voiceA

Get detailed information about a specific ElevenLabs voice by its ID.

Inputs: voice_id (required).

list_modelsA

List all available ElevenLabs TTS models with their capabilities and supported languages.

Inputs: no named parameters.

list_voicesA

List all voices available in ElevenLabs. Returns voice IDs, names, and labels.

Inputs: no named parameters.

speech_to_textA

Transcribe an audio file to text using ElevenLabs Speech-to-Text (scribe_v1). Supports WAV, MP3, and other common formats.

Inputs: file_path (required), model_id, language_code.

text_to_speechA

Convert text to speech using ElevenLabs. Saves the audio as an MP3 file and returns the file path.

Inputs: text (required), model_id, voice_id, stability, +2 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 7 tools

    Disambiguation5/5

    Each tool targets a distinct operation: listing vs fetching voices, listing models, text-to-speech, speech-to-text, user info, and history. There is no overlap or ambiguity between them.

    Naming Consistency4/5

    Most tools follow a clear verb_noun pattern like list_voices, get_voice, get_user_info, and get_history. However, text_to_speech and speech_to_text deviate from this pattern as noun-to-noun phrases, though they are internally consistent and readable.

    Tool Count5/5

    Seven tools is a well-scoped number for a voice/audio agent, covering core operations without bloat. The count is appropriate for the apparent purpose.

    Completeness3/5

    The set covers TTS, STT, voice listing, model listing, user info, and history, but notably lacks a voice-to-voice conversion tool despite the server name. Voice editing/deletion and history item management are also missing, leaving some workflows incomplete.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers