voice-dialog

An MCP server that adds voice conversation capabilities to AI Agents, also usable as a command-line tool. When the Agent finishes its work, it can speak to tell you, and it can also ask you questions and listen to your answers. Speech recognition runs on local whisper, and synthesis uses the system's built-in TTS. No internet connection and no audio transmission throughout.

What it can do

你（没看屏幕）："搞定了吗？"
Agent（出声）："代码写完了，测试全过。要提交吗？"      ← ask_by_voice
你："先不提交，我看看再说"                             ← listen 收到，本地转写
Agent（出声）："好，先留着。"                          ← speak

Related MCP server: Pipecat MCP Server

Quick start

git clone https://github.com/VictorHuang0843/voice-dialog.git
cd voice-dialog

# 没装 uv？先来这个（Windows 用 install.ps1）：
#   curl -LsSf https://astral.sh/uv/install.sh | sh

uv run voice-dialog init

init is a 7-step fully automated guide: install uv → install dependencies (uv even installs Python for you) → download the whisper model (464MB, automatically uses hf-mirror on Chinese networks) → microphone permission test → TTS sound test → print the MCP registration command. If any step fails, it gives you the fix command for that step; just fix it and re-run.

Try a phrase:

uv run voice-dialog speak "语音系统就绪" --lang zh
uv run voice-dialog ask "请说话" --wait-start 60
# 你会听到：高亮上扬"叮↑"= 开始说话 → 停 3 秒 → 低沉"咚…咚↓"= 录音结束

Connect to your Agent

Any MCP client (Claude Code, Codex, Cursor, LoopX...):

{
  "mcpServers": {
    "voice-dialog": {
      "command": "uv",
      "args": ["--project", "/你的路径/voice-dialog", "run", "voice-dialog", "serve"]
    }
  }
}

One-line registration in Claude Code:

claude mcp add voice-dialog -s user -- uv --project /你的路径/voice-dialog run voice-dialog serve

Tools that don't support MCP: call the CLI directly, uv run voice-dialog speak/listen/ask "...".

Claude Code Skill (optional, teaches the Agent when to speak/when to listen): copy skills/voice-dialog/ to ~/.claude/skills/.

Supported platforms

Language auto-detection (whisper supports ~100 languages), VD_LANG=zh can pin it; model size adjustable via VD_MODEL=small.

FAQ

Privacy

Speech is recognized locally (faster-whisper), text is played via system TTS. No cloud APIs are called, and no audio is uploaded.

License

MIT

Model notes

What is downloaded by default? whisper small (int8 quantized, ~464MB), the sweet spot between Chinese/English recognition quality and speed. It downloads automatically on first listen/init, then caches locally (macOS/Linux: ~/.cache/huggingface/hub/, Windows: %USERPROFILE%\.cache\huggingface\hub\), never downloaded again.

How is the download source chosen? The download logic isn't hardcoded in the code — when constructing WhisperModel("small", ...), the faster-whisper library automatically downloads from HuggingFace. It defaults to the official source huggingface.co; if the official source is unreachable, the code automatically switches to the domestic mirror hf-mirror.com and retries (implemented by setting the HF_ENDPOINT environment variable). Chinese users can also pin it manually in advance:

export HF_ENDPOINT=https://hf-mirror.com   # 加进 ~/.zshrc 一劳永逸

Want a larger model? Controlled by the environment variable VD_MODEL, takes effect on re-run:

VD_MODEL=base uv run voice-dialog listen   # 单次用 base

Install from zip (no git clone needed)

Download the zip (or have a friend send it to you) → extract it wherever you like, e.g. ~/tools/ → open a terminal and enter the directory, then let the AI take it from there:

In Claude Code (or any AI coding tool that can execute commands), send it the following verbatim:

帮我安装这个目录里的 voice-dialog 项目：
1. cd 到这个目录跑 uv run voice-dialog init（没装 uv 就先装：curl -LsSf https://astral.sh/uv/install.sh | sh）
2. init 全绿后，把打印出来的 claude mcp add 命令执行掉
3. 最后跑 uv run voice-dialog doctor 给我看结果

It will install everything and register it with Claude Code. Restart the session and it's ready to use.

You can also do it manually, just two commands:

cd 解压后的目录
uv run voice-dialog init        # 结束时打印注册命令，复制执行

If you're not familiar with the terminal on Mac: after extracting, right-click the folder → Services → New Terminal at Folder, and you're in the directory.