Skip to main content
Glama

mcp-ear 👂

An ear for AI assistants. Language models like Claude can talk about sound, and some can even generate it — but most can't listen. mcp-ear is an MCP server that gives Claude (or any MCP client) the ability to listen to audio files on your computer:

  • 🗣️ Hear speech — transcribe it with a local Whisper model (90+ languages, offline, free)

  • 🎵 Hear music — tempo, beat, pitch range, melody notes, musical key

  • 🔊 Hear sound — loudness, dynamics, silences, frequency balance, noise vs. tone

  • 👁️ See sound — a spectrogram image, so a vision-capable model can literally look at what it sounds like

Everything runs on your own machine. No API keys, no audio leaves your computer.

Türkçe README

What it looks like

You: What's in C:\recordings\voice-note.wav?

Claude: (calls listen) It's a 7-second voice note: "Hello Claude. This is a test of your new ear. The quick brown fox jumps over the lazy dog." (calls spectrogram) The speaker has a fairly low, steady voice (around 180 Hz) with three clear phrases separated by short pauses.

The spectrogram tool returns an image like this (waveform on top; time → right, pitch ↑ up, brightness = loudness; cyan line = detected pitch):

spectrogram example

Related MCP server: media-context-mcp

Tools

Tool

What it does

audio_info

Duration, sample rate, channels, format

listen

Speech or sung lyrics → text with timestamps (optional per-word timing), auto language detection. mode: speech, music or auto (switches to music mode when the voice filter would discard most of a song). Known Whisper hallucinations ("Thanks for watching", subtitle credits) are filtered out

analyze

Loudness & dynamics, silences, tempo & beat steadiness, pitch range & common notes, key estimate, energy per frequency band, plain-language hints

melody

The note sequence (e.g. C4 D4 E4 F4 G4) with timing and tuning in cents

spectrogram

PNG image: waveform + mel spectrogram + pitch contour

All tools take an absolute file path, and most accept start / duration (seconds) to focus on part of a long file.

Formats: WAV, FLAC, OGG and MP3 work out of the box. M4A, AAC, OPUS, WEBM and video files (MP4, MOV…) need ffmpeg on your PATH.

Install

You need Python 3.10+. The easiest way is uv, which installs and runs it in one step.

Claude Code

claude mcp add ear -- uvx --from git+https://github.com/mrtdemissek/mcp-ear mcp-ear

Claude Desktop

Open Settings → Developer → Edit Config and add this to claude_desktop_config.json:

{
  "mcpServers": {
    "ear": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/mrtdemissek/mcp-ear", "mcp-ear"]
    }
  }
}

Restart Claude Desktop. Then ask it to listen to a file by its full path.

Without uv

pip install git+https://github.com/mrtdemissek/mcp-ear

Then use mcp-ear as the command in the configs above (no args).

First run: the dependencies are a few hundred MB, and the first listen call downloads the Whisper model (small ≈ 480 MB). After that it works offline.

Settings (environment variables)

Variable

Default

Meaning

MCP_EAR_WHISPER_MODEL

small

tiny, base, small, medium, large-v3, large-v3-turbo. Bigger = more accurate, slower. For songs and non-English speech, medium or larger is noticeably better

MCP_EAR_DEVICE

auto

auto, cpu or cuda. With auto, a GPU is used when available; if the CUDA libraries are missing it falls back to CPU by itself

MCP_EAR_COMPUTE_TYPE

default (int8 on CPU)

Passed to faster-whisper

In Claude Desktop, set them with an "env": { "MCP_EAR_WHISPER_MODEL": "medium" } block next to "args".

Honest limitations

  • It analyzes sound; it doesn't experience it. The plain-language hints ("speech-like", "noise-like", "steady pulse") are heuristics, and the model is told to treat them as guesses.

  • Sung lyrics are much harder than speech: expect misheard words, especially with the small model. In our test on a live acoustic Turkish song, medium got most lines right or nearly right but still misheard some words.

  • melody works best on a single voice or instrument. On a full band mix you get the most prominent line, not a full transcription.

  • Key and tempo estimates can be off by an octave (60 vs 120 BPM) or land on the relative key (C major vs A minor).

  • It doesn't yet recognize sound events (dog bark, doorbell, applause), separate speakers, or listen to a live microphone. See the roadmap.

Roadmap

  • Sound event recognition (e.g. YAMNet/PANNs: "dog barking", "glass breaking", "applause")

  • Speaker diarization ("who spoke when")

  • Live microphone listening

  • Compare two recordings (A/B)

  • PyPI release (uvx mcp-ear)

Contributions are welcome. Open an issue or PR.

Development

git clone https://github.com/mrtdemissek/mcp-ear
cd mcp-ear
python -m venv .venv
.venv/bin/pip install -e ".[dev]"      # Windows: .venv\Scripts\pip
.venv/bin/pytest

The tests synthesize sounds with known answers (an A4 note, a C major scale, a 120 BPM click track, white noise) and check that the ear hears them correctly.

How it works

License

MIT

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to analyze audio files, extracting tempo, key, beat drops, volume surges, high tones, loudness, brightness, and structure, and returning structured JSON and visualizations.
      2
      MIT