Skip to main content
Glama
sandraschi

midasheng-gen-mcp

by sandraschi
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
Python
Local

One call in Claude, Cursor or opencode - "a comedy club scene with a punchline, crowd laughter and a jazz sting" - and you get a single 16 kHz WAV with all of it. Speech lands close to dedicated TTS quality in 9 languages with emotion control, and the whole ~6 GB model stays on your GPU until you unload it.

What this wraps

MiDashengLM-Gen - the first end-to-end trained general text-to-audio model (per the paper): an LLM drives per-token flow matching to generate variable-length audio scenes with speech intelligibility approaching dedicated TTS (Seed-TTS WER 12.15% -> 2.79% vs 1.24% for dedicated TTS). Supports 9 languages and emotion control. Checkpoint auto-downloads from Hugging Face (~6 GB). Model weights are never bundled - see docs/WRAPPEE.md.

Features

How it runs: a local FastMCP 3.4 server (stdio for Claude Desktop, HTTP /mcp for Cursor/webapp) with a React dashboard. The model loads lazily on first generate and stays on the GPU until unloaded.

Direction

Artifacts

Notes

Hands-in

structured caption (caption/asr/speech/sfx/music/env views), guidance + seed params

Via MCP tool, REST, or webapp

Hands-out

16 kHz mono WAV scenes, indexed in SQLite, browsable/exportable

audio_scene(operation="export") or webapp

  • Generate whole soundscapes in one call - crowd laughter + jazz sting + comedy speech, or rain + thunder + forest ambience

  • Speech intelligibility near dedicated TTS quality, 9 languages, emotion control through the speech view

  • Scene library with pagination, audio playback, and export in the webapp

  • Async job API for long generations; Prefab UI cards in chat

  • Variable-length output via the learned stop head - no fixed-duration cuts

Quick Install

The fastest path is the .mcpb bundle for Claude Desktop (see INSTALL.md for all options):

  1. Download midasheng-gen-mcp-v0.1.0.mcpb from Releases

  2. Drag it onto Claude Desktop

  3. First use will download the ~6 GB checkpoint automatically

Or clone and double-click start.bat for the full stack (backend + webapp).

Example Prompts

  • "Generate a comedy club scene: a punchline, crowd laughter, and a jazz band sting" -> audio_scene(operation="generate", caption="A comedian delivering a punchline followed by uproarious crowd laughter", asr="And that is why I never buy cheap luggage anymore!", speech="expressive comedic male voice", music="sudden upbeat jazz band sting", sfx="crowd laughter", env="intimate comedy club")

  • "Make a thunderstorm at night in a forest" -> audio_scene(operation="generate", caption="A rolling thunderstorm in a forest at night", sfx="distant thunder and heavy rain", env="dense forest", seed=42)

  • "What's the model state?" -> audio_scene(operation="status")

Documentation

Doc

Contents

Installation

All install methods, prerequisites

Onboarding

First-run model download, GPU checks, pitfalls

Wrapped app

MiDashengLM-Gen paper, weights, license, demo

Architecture

System architecture, ports, data flow

Configuration

Env vars, config options

Tool Reference

All available tools

Development

Contributing, local setup

Troubleshooting

Common issues

Requirements

  • Windows/Linux/macOS with a CUDA GPU (RTX 4090 class: ~12 GB VRAM (fp32); CPU inference works but is slow)

  • Python 3.12+ via uv, Node.js 20+, bun (auto-installed by start.ps1 on naked PCs)

  • ~10 GB free disk (checkpoint ~6 GB + torch stack)

License

Apache-2.0 (model and this wrapper). See the use-restrictions section in the upstream repo: no unlawful/military use, no harm to minors or groups.