One line: voice-analysis-mcp is a fully local, no-API-key MCP server that lets an LLM qualitatively review call recordings, voice memos, TTS output, and other audio (or the audio track of video) via transcription, conversation dynamics, vocal delivery, recording quality, and visual inspection.
Inspect files (get_audio_info): duration, channels, sample rate, codec, tags — the starting point that drives channel handling and later windowing.
Transcribe (transcribe): local Whisper models (tiny → large-v3-turbo, downloaded on first use), timestamped segments, language detection, words-per-minute, optional word-level timing, per-channel isolation on stereo calls, and segment pagination via max_segments/next_start_time.
Analyze conversation dynamics (analyze_conversation): talk time and turns per speaker, dead air, overlap, interruptions vs. backchannels, and response latency on stereo (per-speaker) calls; overall speech/silence dynamics on mono. Cheap whole-file pass.
Analyze prosody (analyze_prosody, ≤300 s windows): pitch median/range (monotone vs. expressive), loudness and dynamic range, syllable rate, pause/hesitation patterns, optionally per channel.
Assess recording quality (analyze_quality): levels (overall and per channel), clipping, noise floor, estimated SNR, effective bandwidth (catches upsampled telephone audio), channel imbalance, flagged issues.
Extract moments (extract_segment): cut a time range to a standalone wav (never overwrites unless overwrite=true).
Visualize (render_spectrogram, ≤600 s; render_waveform, ≤600 s): mel spectrogram images to spot hold music, DTMF/beeps, hum, dropouts, TTS glitches; per-channel waveform rows for speaker activity, silences, clipping, and level imbalance.
Typical flow: info → conversation → transcribe (per channel) → prosody/quality/spectrogram on flagged windows; all timestamps are absolute seconds so findings from different tools line up, results are compact JSON sized for the context window, and images come back as MCP image content the model can view.
Runs anywhere locally: ffmpeg decode, faster-whisper transcription, librosa analysis; CPU int8 by default, optional GPU via VOICE_ANALYSIS_WHISPER_DEVICE=cuda and compute-type env vars; tools run non-blocking in worker threads.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@voice-analysis-mcp Review ~/calls/support-0412.wav: who dominated the conversation?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that gives LLMs like Claude the tools to review and do qualitative analysis of call recordings, voice memos, TTS output, and other audio files — transcription, conversation dynamics, vocal delivery, recording quality, and visual inspection via spectrograms/waveforms.
Everything runs locally: ffmpeg for decoding, faster-whisper for transcription (models download on first use), librosa for signal analysis. No API keys required.
Tool | What it answers |
| What is this file? Duration, channels, sample rate, codec, tags. |
| What was said, when? Timestamped segments, optional word timing, words-per-minute. |
| Who talks how much? Talk time and turns per channel, dead air, overlap, interruptions vs. backchannels, response latency (stereo calls: one speaker per channel). |
| How does it sound? Pitch median/range (monotone vs expressive), loudness and dynamic range, estimated syllable rate, pause/hesitation patterns for a window (≤ 300 s). |
| Can I trust this recording? Levels (overall and per channel), clipping, noise floor, estimated SNR, effective bandwidth (catches upsampled telephone audio), channel imbalance, flagged issues. |
| Cut a key moment out to a standalone wav file. Never overwrites unless |
| A mel spectrogram image the model can look at — spot hold music, DTMF/beeps, hum, dropouts, TTS artifacts. |
| Per-channel waveform image — speaker activity, silences, level imbalance at a glance. |
A typical call review: get_audio_info → analyze_conversation (whole file,
cheap) → transcribe (per channel for stereo) → analyze_prosody /
render_spectrogram on the interesting windows.
Related MCP server: ollos-mcp
Python 3.11–3.13 and uv
ffmpeg on PATH (brew install ffmpeg)
uv sync
uv run pytest # optional: verify (transcription tests skip unless the tiny Whisper model is cached)
uv run ruff check . # lintWorking inside this repo, nothing to configure: the checked-in .mcp.json
registers the server automatically (approve it when prompted). From any other
project, register it globally:
claude mcp add --scope user voice-analysis -- uv run --directory /path/to/voice-analysis.mcp voice-analysis-mcpclaude_desktop_config.json){
"mcpServers": {
"voice-analysis": {
"command": "uv",
"args": ["run", "--directory", "/path/to/voice-analysis.mcp", "voice-analysis-mcp"]
}
}
}The model only needs a file path — every tool takes an absolute path to a local audio file, so "analyze the call in ~/Downloads/call-4711.mp3" is enough to start. The server's instructions steer the model through the intended flow:
get_audio_info — learn duration and channel count (drives everything else)
analyze_conversation — cheap whole-file pass: talk balance, dead air, interruptions
transcribe — content, per channel on stereo calls for who-said-what
analyze_prosody / analyze_quality / render_spectrogram on windows the
first passes flagged — all results use absolute seconds, so findings from
different tools line up
Example prompts that work end-to-end:
Review ~/calls/support-0412.wav: who dominated the conversation, were there awkward silences, and how did the agent's tone change after the customer pushed back?
Compare the TTS output in v1.wav and v2.wav — which sounds less monotone and are there any audible artifacts?
Results are compact JSON (rounded numbers, capped lists) sized for a model's context window; images come back as MCP image content Claude can view directly.
Stereo = per-speaker. Telephony platforms typically export calls with one
speaker per channel. Tools take a channel parameter to isolate a speaker;
analyze_conversation uses both channels for interruption/overlap/latency
metrics. Mono recordings fall back to overall speech/silence dynamics —
model-based diarization (e.g. pyannote) is a possible future addition.
Whisper models: tiny/base/small/medium/large-v3/large-v3-turbo,
cached under the Hugging Face cache dir after first download. base is the
default trade-off; use small+ for noisy telephone audio. Inference runs on
CPU with int8 weights by default; set VOICE_ANALYSIS_WHISPER_DEVICE=cuda
and VOICE_ANALYSIS_WHISPER_COMPUTE_TYPE=float16 for a GPU.
Long transcripts are capped at max_segments (default 400). The
response then carries truncated: true and next_start_time; call again
from there.
Non-blocking tools. Every tool runs its CPU work in a worker thread, so the server keeps answering pings and cancellations during a long transcription.
Windows over whole files for the expensive analyses: prosody is capped at
300 s and images at 600 s per call — pass start_time/end_time. All
timestamps in results are absolute seconds into the file, so findings from
different tools line up. Windows outside the file are rejected rather than
silently clamped.
Energy-based VAD drives the conversation, prosody and quality metrics. It adapts to the recording's noise floor, treats overlaps under 0.3 s as backchannels rather than interruptions, and only counts a response latency when the first speaker really finished (no resume before the reply).
Any input format works if ffmpeg can decode it, including pulling the audio track out of video files.
Measure conversation dynamics: talk time per channel, turns, dead air, overlap.
On stereo call recordings (one speaker per channel) this also reports
interruption counts per side, overlap ratio, and response latency —
who dominates, who interrupts, how quickly each side responds. On mono
audio only overall speech/silence dynamics are available. Cheap to run
on a whole recording; a good early step for call review.
Inputs: path (required), end_time, start_time.
Measure vocal delivery in a window (max 300s): pitch, energy, pace, pausing.
Returns pitch statistics (median, range in semitones — low range means
monotone delivery), loudness/dynamic range, speech ratio, and pause
patterns (hesitations). Use on specific moments found via transcription or
conversation analysis — e.g. compare an agent's delivery early vs late in a
call, or check whether TTS output sounds flat. Pass channel to isolate one
speaker on stereo recordings.
Inputs: path (required), channel, end_time, start_time.
Assess recording quality: levels, clipping, noise floor, estimated SNR.
Flags common problems (clipping, very low level, high background noise,
narrowband telephone audio). Run this before drawing conclusions from other
analyses — poor quality audio degrades transcription and pitch tracking.
Inputs: path (required), end_time, start_time.
Cut a time range out of a recording into a standalone wav file.
Useful for isolating a key moment (an objection, an escalation, a TTS
artifact) to share or analyze further. Returns the path of the new file;
if output_path is omitted the file is written next to the source.
Inputs: path (required), end_time (required), start_time (required), output_path.
Get metadata for an audio/video file: duration, channels, sample rate, codec, tags.
Call this first for any new file — duration tells you how to window later
analyses, and channels tells you whether per-speaker (stereo) analysis is possible.
Inputs: path (required).
Render a mel spectrogram image of a window (max 600s) for visual inspection.
Look at this to spot things metrics miss: hold music, DTMF/beep tones,
hum, dropouts, TTS glitches, where energy concentrates. Time axis is in
absolute seconds so findings map back to transcript timestamps.
Inputs: path (required), channel, end_time, start_time.
Render a waveform image (max 600s), one row per channel.
On stereo call recordings the two rows show each speaker's activity at a glance — useful for spotting long silences, who talks when, clipping, and level imbalance between sides.
Inputs: path (required), end_time, start_time.
Transcribe speech to timestamped text using a local Whisper model (no API key).
Returns segments with start/end times plus detected language and overall
words-per-minute. On stereo call recordings, pass channel=0 or channel=1 to
transcribe one speaker at a time and interleave the results by timestamp for
an accurate who-said-what transcript. model_size: tiny/base/small/medium/
large-v3/large-v3-turbo — larger is more accurate but slower; the model is
downloaded on first use. word_timestamps=True adds per-word timing (useful
for locating exact moments, at some cost in output size).
Inputs: path (required), channel, end_time, language, +3 more.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 8 tools
Each tool targets a distinct aspect of voice analysis: metadata, transcription, conversation dynamics, prosody, quality, segment extraction, and two visualizations. Descriptions clearly differentiate their purposes, leaving no ambiguity.
Most tools follow a consistent verb_noun pattern (get_audio_info, analyze_conversation, render_spectrogram), but 'transcribe' is a single verb outlier, slightly breaking the pattern. Overall, naming is mostly predictable and clear.
With 8 tools, the server covers a well-scoped range of voice analysis capabilities—from metadata to transcription to advanced metrics and visualizations—without unnecessary redundancy or bloat.
Core workflows for audio analysis are covered (info, transcription, conversation dynamics, prosody, quality, segment extraction, visualizations). A minor gap is lack of explicit speaker diarization beyond stereo transcription, but the set is largely complete for typical analysis tasks.
Audio and video tools for your AI assistant: remove noise, enhance voices, transcribe, and more.
Search speech in podcasts, government meetings, and your own audio: speakers, entities, timestamps.
Audio AI tools: transcription with speakers, stems, noise removal, MIDI, speech, voices and music.
Transcribe audio & video to text for AI agents: 100+ languages, speaker labels, webhooks.
