Talky Talky
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Talky TalkySay 'Hello, how can I help you?' in a friendly voice."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Talky Talky
A comprehensive audio MCP (Model Context Protocol) server for AI agents. Generate speech, transcribe audio, clone voices, analyze speech quality, design soundscapes, and manage audio assets—all through a standardized interface that works with any MCP-compatible client.
Quick Setup with AI Agents
Copy and paste this prompt to your AI agent (Claude Code, Cursor, Windsurf, etc.) to have it automatically configure Talky Talky:
Install and configure the Talky Talky MCP server for audio capabilities.
1. Clone the repo: git clone https://github.com/shawnrushefsky/talky-talky.git
2. Find the full path to uv: which uv (e.g., /Users/username/.local/bin/uv)
3. Add to my MCP configuration:
For Claude Desktop (GUI app - MUST use full path to uv):
{
"mcpServers": {
"talky-talky": {
"command": "/full/path/to/uv",
"args": ["run", "--directory", "<path-to-talky-talky>", "--extra", "macos-full", "talky-talky"]
}
}
}
For CLI tools (Claude Code, .mcp.json):
{
"mcpServers": {
"talky-talky": {
"command": "uv",
"args": ["run", "--directory", "<path-to-talky-talky>", "--extra", "tts", "--extra", "transcription", "--extra", "analysis", "talky-talky"]
}
}
}
4. Replace paths with actual values
5. Restart the application and verify by checking TTS availability
Platform extras:
- macOS: macos-full (TTS + transcription + analysis, excludes CUDA-only engines)
- Linux with CUDA: linux-cuda-full (all engines including CUDA-only)
- CPU only: cpu-full
Requirements: Python 3.11+, ffmpeg, GPU recommended for TTS/transcription engines.Related MCP server: Claud-Ear
Table of Contents
Features
Text-to-Speech (11 Engines)
Engine | Description | Languages | License |
Maya1 | Voice design from natural language descriptions | Multi | Apache-2.0 |
Chatterbox | Voice cloning with emotion control | 23 | MIT |
Chatterbox Turbo | Fast voice cloning for production | 23 | MIT |
MiraTTS | Ultra-fast cloning at 100x realtime (CUDA) | Multi | MIT |
XTTS-v2 | Cross-language voice cloning | 17 | CPML |
Kokoro | 54 pre-built voices, lightweight | 8 | Apache-2.0 |
Soprano | 2000x realtime speed (CUDA) | EN | Apache-2.0 |
VibeVoice Realtime | ~300ms latency real-time TTS | EN | MIT |
VibeVoice Long-form | Multi-speaker up to 90 min | EN/ZH | MIT |
CosyVoice3 | Instruction-controlled cloning | 9 | Apache-2.0 |
SeamlessM4T v2 | Translation + TTS, 200 speakers | 35 | CC-BY-NC-4.0 |
Song Generation (2 Engines)
Engine | Description | Platform | License |
LeVo | Complete songs from lyrics with vocals | CUDA | MIT |
ACE-Step | Foundation model, style prompts + lyrics | CUDA/MPS | MIT |
Speech-to-Text (2 Engines)
Engine | Description | Languages | Speed |
Whisper | OpenAI's robust ASR via transformers | 99+ | 1x |
Faster-Whisper | CTranslate2-optimized Whisper | 99+ | 4x faster |
Audio Analysis
Speech Quality: MOS prediction, noisiness, discontinuity, coloration
Emotion Detection: 9 emotions (angry, happy, sad, surprised, etc.)
Voice Similarity: Speaker verification and comparison
TTS Verification: Automated quality checks for generated audio
SFX Analysis
Loudness: Peak, RMS, LUFS, dynamic range, true peak
Clipping Detection: Find digital distortion regions
Spectral Analysis: Frequency content, brightness, energy distribution
Silence Detection: Leading/trailing silence, gaps, content boundaries
Audio Design
Mixing: Layer multiple tracks with volume control
Effects: EQ (lowpass, highpass, bass, treble), reverb, echo, speed
Fades: Fade in/out with configurable duration
Overlays: Position-based audio layering
Crossfades: Smooth transitions between segments
Trimming: Auto-detect content boundaries, remove silence
Silence Insertion: Add controlled pauses between segments
Voice Modulation
Pitch Shifting: Change pitch without affecting speed (±12 semitones)
Time Stretching: Change speed without affecting pitch
Voice Effects: Robot, chorus, vibrato, flanger, telephone, and more
Formant Shifting: Change voice character (masculine/feminine)
Note: Maya1 can achieve pitch, pacing, and timbre variations directly through voice descriptions (e.g., "high pitch, fast pacing, gravelly timbre"). Use the voice modulation tools for post-processing or with engines that don't have built-in voice control.
Audio Asset Management
Local Library: Index and search local folders with SQLite FTS5
Freesound.org: Search and download CC-licensed sounds
Jamendo: 500k+ CC-licensed music tracks for free (non-commercial)
License Tracking: CC0, CC-BY, CC-BY-NC, CC-BY-SA attribution
Tagging: Manual and AI-powered auto-tagging
Audio Utilities
Format Conversion: WAV, MP3, M4A with ffmpeg
Concatenation: Join files with optional gaps
Normalization: Broadcast standard (-16 LUFS)
Playback: System default audio player
Cross-Platform Support
CUDA: Full support with GPU acceleration
MPS: Apple Silicon support for most engines
CPU: Fallback for all engines (slower)
Installation
Prerequisites
Python 3.11+ (required for TTS library compatibility)
ffmpeg (required for audio processing)
GPU (recommended for TTS and transcription)
Don't have Python 3.11+? Use uv which auto-manages Python versions:
curl -LsSf https://astral.sh/uv/install.sh | sh uv run --extra tts talky-talky
Platform-Specific Installation
git clone https://github.com/shawnrushefsky/talky-talky.git
cd talky-talky
# macOS (Apple Silicon or Intel)
pip install -e ".[macos-full]" # TTS + transcription + analysis
# Linux with NVIDIA CUDA GPU
pip install -e ".[linux-cuda-full]" # All engines including CUDA-only
# CPU only (no GPU)
pip install -e ".[cpu-full]" # Excludes CUDA-only enginesIndividual Components
# TTS engines
pip install -e ".[maya1]" # Voice design
pip install -e ".[chatterbox]" # Voice cloning (includes Turbo)
pip install -e ".[xtts]" # Multilingual cloning
pip install -e ".[kokoro]" # Pre-built voices
pip install -e ".[seamlessm4t]" # Multilingual + translation
pip install -e ".[mira]" # Fast cloning (CUDA only)
pip install -e ".[soprano]" # Ultra-fast (CUDA only)
# Transcription
pip install -e ".[whisper]" # OpenAI Whisper
pip install -e ".[faster-whisper]" # 4x faster Whisper
# Analysis
pip install -e ".[emotion2vec]" # Emotion detection
pip install -e ".[resemblyzer]" # Voice similarity
pip install -e ".[nisqa]" # Speech quality
# Song Generation (CUDA required for LeVo)
pip install -e ".[songgen]" # LeVo + ACE-Step
# Combined
pip install -e ".[tts]" # All TTS engines
pip install -e ".[transcription]" # All transcription engines
pip install -e ".[analysis]" # All analysis enginesUsing uv (Recommended)
git clone https://github.com/shawnrushefsky/talky-talky.git
cd talky-talky
# Run directly without installing
uv run --extra macos-full talky-talkyUsing Docker
# Basic image (audio utilities only)
docker pull ghcr.io/shawnrushefsky/talky-talky:latest
docker run -i ghcr.io/shawnrushefsky/talky-talky:latest
# With GPU access
docker run -i --gpus all talky-talky-cudaConfiguration
Claude Desktop
Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS):
{
"mcpServers": {
"talky-talky": {
"command": "/full/path/to/uv",
"args": ["run", "--directory", "/path/to/talky-talky", "--extra", "macos-full", "talky-talky"]
}
}
}macOS Note: GUI apps don't inherit shell PATH. Use full path to uv (
which uv).
Claude Code / CLI
Add to .mcp.json or ~/.claude/settings.json:
{
"mcpServers": {
"talky-talky": {
"command": "uv",
"args": ["run", "--directory", "/path/to/talky-talky", "--extra", "tts", "--extra", "transcription", "talky-talky"]
}
}
}Available Tools
TTS Engine Tools
Tool | Description |
| Check available engines and device info |
| Get detailed engine info and parameters |
| List installed engines |
| Check Maya1 model download status |
| Download Maya1 models (~10GB) |
Speech Generation Tools
Tool | Description |
| Generate with voice description |
| Voice cloning with emotion control |
| Fast voice cloning |
| Fast cloning, 48kHz (CUDA) |
| Multilingual cloning (17 languages) |
| Pre-built voices (54 voices) |
| Ultra-fast TTS (CUDA) |
| Real-time TTS |
| Long-form multi-speaker |
| Instruction-controlled cloning |
| Multilingual TTS + translation |
Song Generation Tools
Tool | Description |
| Check song generation engine status |
| Get engine details and requirements |
| List installed engines |
| Check LeVo model download status |
| Download LeVo models (~10GB) |
| Get lyrics format guide |
| Generate songs from lyrics (CUDA) |
| Check ACE-Step model status |
| Download ACE-Step models (~7GB) |
| Generate songs with ACE-Step (MPS/CUDA) |
Transcription Tools
Tool | Description |
| Check transcription engine status |
| Get engine details |
| List installed engines |
| Transcribe audio to text |
| Transcribe with word-level timing |
| Verify TTS matches expected text |
Audio Analysis Tools
Speech Analysis
Tool | Description |
| Check analysis engine status |
| Get engine details |
| Detect emotion (9 categories) |
| Compare voices for similarity |
| Get voice embedding vector |
| MOS score and quality dimensions |
| Combined quality verification |
SFX Analysis
Tool | Description |
| Check SFX tools availability |
| Peak, RMS, LUFS, dynamic range |
| Find clipped samples/regions |
| Frequency and energy analysis |
| Find silence regions and gaps |
| Validate against target specs |
Audio Design Tools
Tool | Description |
| Layer multiple tracks together |
| Volume control (multiplier or dB) |
| Fade in/out effects |
| EQ, reverb, echo, speed |
| Position-based audio overlay |
| Smooth transitions between clips |
| Trim with auto silence detection |
| Add controlled pauses |
| Batch silence detection |
Voice Modulation Tools
Tool | Description |
| Change pitch without affecting speed |
| Change speed without affecting pitch |
| Apply voice effects (robot, chorus, etc.) |
| List available voice effect presets |
| Change voice character (masculine/feminine) |
Audio Utility Tools
Tool | Description |
| Get duration, format, size |
| Convert WAV/MP3/M4A |
| Concatenate with optional gaps |
| Normalize to -16 LUFS |
| Check ffmpeg installation |
| Play with system player |
| Set default output path |
| Get current output path |
Audio Asset Management Tools
Tool | Description |
| List sources (local, Freesound, Jamendo) |
| Search SFX, music, ambience |
| Get asset details |
| Download to local storage |
| Import folder to library |
| Set Freesound API key |
| Set Jamendo client ID for music |
| Add tags to asset |
| Remove tags from asset |
| List all tags with counts |
| AI-powered auto-tagging |
| Check auto-tag engines |
TTS Engine Guide
Maya1 (Voice Design)
Create unique voices from natural language descriptions with 20+ emotion tags.
speak_maya1(
text="I can't believe it! <laugh> We did it!",
output_path="output.wav",
voice_description="Excited young woman, American accent, energetic"
)Emotion Tags: <laugh> <chuckle> <sigh> <gasp> <whisper> <angry> <yell> <cry> <cough>
Chatterbox (Voice Cloning)
Zero-shot voice cloning with emotion control. 23 languages.
speak_chatterbox(
text="Hello! [chuckle] Nice to meet you.",
output_path="output.wav",
reference_audio_paths=["reference.wav"],
exaggeration=0.6 # 0.0-1.0+ for expressiveness
)SeamlessM4T v2 (Multilingual + Translation)
Generate speech in 35 languages with optional translation.
# Pure TTS
speak_seamlessm4t(
text="Hello world",
output_path="hello.wav",
language="en"
)
# Translate English to French speech
speak_seamlessm4t(
text="Hello world",
output_path="bonjour.wav",
src_language="en",
language="fr",
speaker_id=5 # 200 speaker options
)Kokoro (Pre-built Voices)
54 high-quality voices, 8 languages, lightweight (82M).
speak_kokoro(
text="Welcome!",
output_path="output.wav",
voice="af_heart", # American female
speed=1.0
)Voice Format: [lang][gender]_[name] (e.g., af_heart, bm_george, jf_alpha)
See CLAUDE.md for complete documentation on all engines.
Transcription Engine Guide
Faster-Whisper (Recommended)
4x faster than standard Whisper with same accuracy.
transcribe_audio(
audio_path="speech.wav",
engine="faster_whisper",
model_size="large-v3", # tiny, base, small, medium, large-v3
language="en" # or None for auto-detection
)With Timestamps
transcribe_with_timestamps(
audio_path="speech.wav",
word_level=True # Get word-level timing
)TTS Verification
verify_tts_output(
audio_path="generated.wav",
expected_text="Hello world",
similarity_threshold=0.8
)Usage Examples
Generate Narration with Custom Voice
Generate speech saying "Welcome to the future of AI" with a deep male narrator voice using Maya1Clone a Voice
Clone the voice from sample.wav using Chatterbox and say "This is voice cloning" with high expressivenessTranscribe Audio
Transcribe the audio file interview.wav with word-level timestampsCreate a Podcast Intro
1. Generate intro music search for "podcast intro" sounds
2. Generate speech: "Welcome to Tech Talk" with a professional voice
3. Mix the music and speech together with music at 20% volume
4. Add fade in/out effectsAnalyze Speech Quality
Analyze the speech quality of generated.wav and check if it meets broadcast standardsTranslate and Speak
Use SeamlessM4T to translate "Hello, how are you?" from English to Japanese speechVoice Modulation
1. Generate speech with Maya1 saying "Hello there"
2. Create a robot version using the robot voice effect
3. Create a deeper version by shifting formants to 0.8
4. Create a chipmunk version by pitching up 6 semitonesGenerate a Song
Generate a pop song with LeVo using these lyrics:
[intro-short] ; [verse] Walking through the city lights. Stars are shining bright tonight ;
[chorus] This is our moment. Nothing can stop us now ; [outro-short]
Use description: "female, pop, upbeat, synth"Generate Music with ACE-Step (Apple Silicon)
Generate an instrumental jazz track with ACE-Step:
- Style prompt: "instrumental, jazz, piano, smooth, relaxing"
- Duration: 60 secondsDevelopment
# Install dev dependencies
pip install -e ".[dev]"
# Run tests
pytest
# Lint and format
ruff check talky_talky
ruff format talky_talkyThis project uses uv for package management.
License
MIT
Individual engines have their own licenses:
Apache-2.0: Maya1, Kokoro, Soprano, CosyVoice3
MIT: Chatterbox, MiraTTS, VibeVoice, LeVo, ACE-Step
CPML: XTTS-v2 (Coqui Public Model License)
CC-BY-NC-4.0: SeamlessM4T v2 (non-commercial only)
Contributing
Contributions welcome! Please open an issue or submit a pull request.
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/shawnrushefsky/talky-talky'
If you have feedback or need assistance with the MCP directory API, please join our Discord server