voice-analysis-mcp

An MCP server that gives LLMs like Claude the tools to review and do qualitative analysis of call recordings, voice memos, TTS output, and other audio files — transcription, conversation dynamics, vocal delivery, recording quality, and visual inspection via spectrograms/waveforms.

Everything runs locally: ffmpeg for decoding, faster-whisper for transcription (models download on first use), librosa for signal analysis. No API keys required.

Tools

A typical call review: get_audio_info → analyze_conversation (whole file, cheap) → transcribe (per channel for stereo) → analyze_prosody / render_spectrogram on the interesting windows.

Related MCP server: voice-mcp-server

Requirements

• Python 3.11–3.13 and uv

• ffmpeg on PATH (brew install ffmpeg)

Setup

uv sync
uv run pytest        # optional: verify

Claude Code

Working inside this repo, nothing to configure: the checked-in .mcp.json registers the server automatically (approve it when prompted). From any other project, register it globally:

claude mcp add --scope user voice-analysis -- uv run --directory /path/to/voice-analysis.mcp voice-analysis-mcp

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "voice-analysis": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/voice-analysis.mcp", "voice-analysis-mcp"]
    }
  }
}

How a Claude instance uses this

The model only needs a file path — every tool takes an absolute path to a local audio file, so "analyze the call in ~/Downloads/call-4711.mp3" is enough to start. The server's instructions steer the model through the intended flow:

1. get_audio_info — learn duration and channel count (drives everything else)

2. analyze_conversation — cheap whole-file pass: talk balance, dead air, interruptions

3. transcribe — content, per channel on stereo calls for who-said-what

4. analyze_prosody / analyze_quality / render_spectrogram on windows the first passes flagged — all results use absolute seconds, so findings from different tools line up

Example prompts that work end-to-end:

Review ~/calls/support-0412.wav: who dominated the conversation, were there awkward silences, and how did the agent's tone change after the customer pushed back?

Compare the TTS output in v1.wav and v2.wav — which sounds less monotone and are there any audible artifacts?

Results are compact JSON (rounded numbers, capped lists) sized for a model's context window; images come back as MCP image content Claude can view directly.

Notes & design choices

• Stereo = per-speaker. Telephony platforms typically export calls with one speaker per channel. Tools take a channel parameter to isolate a speaker; analyze_conversation uses both channels for interruption/overlap/latency metrics. Mono recordings fall back to overall speech/silence dynamics — model-based diarization (e.g. pyannote) is a possible future addition.

• Whisper models: tiny/base/small/medium/large-v3/large-v3-turbo, cached under the Hugging Face cache dir after first download. base is the default trade-off; use small+ for noisy telephone audio.

• Windows over whole files for the expensive analyses: prosody is capped at 300 s and images at 600 s per call — pass start_time/end_time. All timestamps in results are absolute seconds into the file, so findings from different tools line up.

• Any input format works if ffmpeg can decode it, including pulling the audio track out of video files.