JOONS LISTEN MCP

Local meeting transcription for MCP clients: Whisper large-v3 and Qwen3-ASR-1.7B on your own GPU, with speaker labels, fact flags and optional Korean translation and minutes.

Part of the JOONS ecosystem: MCP-Based AI Infrastructure Ecosystem.

Korean: README.ko.md

Status: 0.1.0 Preview. Read LIMITATIONS.md before relying on it.

At a glance

• What: a small local service plus a stdio MCP server. You give it a meeting or call recording; it returns a transcript with speaker turns, timestamps, "check this" flags on numbers, prices, dates and product codes, and optionally a Korean translation and minutes written by a local LLM.

• For: people who record meetings or calls (Chinese, Korean, English, mixed) and want the audio to stay on their own machine, and want an MCP client such as Claude Desktop or Claude Code (anything that can start a local stdio server) or a script to drive it. ChatGPT cannot start local stdio servers; it needs a remote MCP endpoint, so put the service behind an MCP gateway you control.

• Not for: real-time captions, cloud processing, legal-grade transcripts, or machines without a CUDA GPU (it is tested only on an 8 GB NVIDIA card; CPU-only use is untested).

• How: install the dependencies, start the service, point your MCP client at python -m joons_listen mcp, then ask the client to transcribe a file.

• Trust: the service binds to loopback only, reads audio only from folders you allow, and sends meeting text only to a local LLM. Every number it extracts is marked "verify": nothing is auto-confirmed. See SECURITY.md.

Related MCP server: meeting-scribe

What it does

audio file
  -> ffmpeg decode (16 kHz mono)
  -> silero-vad speech segments (gaps the VAD misses are filled, see LIMITATIONS)
  -> speaker turns: ECAPA embeddings + agglomerative clustering
  -> ASR: Qwen3-ASR-1.7B (zh, ko, mixed) or Whisper large-v3 (en), chosen per language
  -> fact extraction (price, quantity, date, code) + uncertainty flags
  -> optional Korean translation and minutes via a local Ollama model
  -> txt / md / srt / json / minutes files

• Two ASR engines only, loaded one at a time so an 8 GB GPU is enough.

• Jobs run one by one in a queue and survive restarts: each stage writes a checkpoint, so a stopped job resumes instead of starting over.

• A private dictionary (names, replacement rules) biases recognition and fixes recurring mistakes.

• Hallucination guards: repeated-text detection, a filter for dictionary hints echoed back as text on noisy audio, low-confidence flags.

Requirements

• Python 3.10 or newer, ffmpeg on PATH (or JOONS_LISTEN_FFMPEG).

• An NVIDIA GPU with roughly 8 GB of memory and a CUDA build of PyTorch.

• Python packages: numpy, soundfile, scikit-learn, torch, faster-whisper, qwen-asr, silero-vad, speechbrain, huggingface_hub.

• Optional, for translation and minutes: Ollama with a local model (default qwen3:8b).

• Model weights (about 10 GB) are downloaded from Hugging Face on first use into the data folder. They are not part of this repository.

Install and run

git clone https://github.com/dbdefault/joons-listen-mcp
cd joons-listen-mcp
pip install -e .
pip install torch faster-whisper qwen-asr silero-vad speechbrain huggingface_hub   # pick the torch build for your CUDA

python -m joons_listen check      # GPU, engines, local LLM
python -m joons_listen serve      # the HTTP service on 127.0.0.1:18796

The package is installed from source; it is not published to PyPI.

Without an MCP client you can process one file directly (stop the service first, both would share one job database):

python -m joons_listen run meeting.m4a --translate --minutes

Connect an MCP client

The MCP server speaks JSON-RPC over stdio and only calls the running service on loopback. Example client configuration (also in examples/mcp-client-config.json):

{
  "mcpServers": {
    "joons-listen": {
      "command": "python",
      "args": ["-m", "joons_listen", "mcp"],
      "env": { "JOONS_LISTEN_URL": "http://127.0.0.1:18796" }
    }
  }
}

Tools:

Long recordings take time (a 56-minute recording took about 20 minutes on an RTX 4060 Ti), so the tools never wait: call listen_status until the state is done, then listen_result.

Allowed folders

The service reads only audio files inside allowed folders. The default is the data folder's inbox, ~/Desktop and ~/Downloads. Set JOONS_LISTEN_ROOTS (a path-separator list) to change it. A bare file name is looked up inside the allowed folders. Files whose names look like secrets are refused.

Configuration

All settings are environment variables. The main ones:

Evaluation

docs/EVALUATION.md has error rates and speed on a public speech set, and a field test on a hard far-field phone recording. tools/eval_fleurs.py and tools/eval_recording.py reproduce them. The measurements are small and meant to find weaknesses, not to rank the two engines.

Documentation

• docs/API.md: HTTP API of the service

• docs/EVALUATION.md: measurements and what they mean

• LIMITATIONS.md: known weaknesses

• SECURITY.md: trust model and reporting

• CHANGELOG.md

Third-party models

Check each upstream license yourself before redistributing weights. This repository contains none.

License

Apache License 2.0, see LICENSE.