Resemble AI Voice Generation MCP Server

# Resemble AI Voice Generation MCP Server A server implementation for the [Resemble AI](https://www.resemble.ai/) voice generation API that integrates with [Claude](https://www.anthropic.com/claude) and [Cursor](https://cursor.sh/) using the Model Context Protocol (MCP). ## Features - Generate voice audio from text using Resemble AI's voices - List available voice models - Return audio as local file or base64 encoded string - Multiple connection methods: - **SSE Transport** - Network-based Server-Sent Events (default) - **StdIO Transport** - Direct process communication ## Setup Instructions ### Prerequisites - Python 3.10 or higher - Resemble AI API key (sign up at [Resemble AI](https://www.resemble.ai/)) ### Environment Setup #### Option 1: Using Conda (Recommended) ```bash # Run the setup script ./scripts/setup_environment.sh # Activate the environment conda activate resemble_mcp ``` #### Option 2: Using Virtual Environment ```bash # Run the setup script ./scripts/setup_venv.sh # Activate the environment source venv/bin/activate ``` ### Configuration Set your Resemble AI API key as an environment variable: ```bash export RESEMBLE_API_KEY="your_api_key_here" ``` Alternatively, create a `.env` file in the project root with the following content: ``` RESEMBLE_API_KEY=your_api_key_here ``` ## Running the Server ### Using the Run Script (Recommended) Choose your preferred implementation: ```bash # Run the MCP SDK implementation with SSE transport (default) ./run_server.sh mcp 8083 # Run the HTTP implementation ./run_server.sh http 8083 # Run with StdIO transport (for direct process communication) ./run_server.sh stdio ``` ### Using the CLI Directly ```bash # Run the MCP SDK implementation with SSE transport python -m src.cli --implementation mcp --port 8083 # Run with StdIO transport python -m src.cli --implementation stdio ``` ## Connecting to Claude Desktop ### SSE Transport Connection Create a `claude_desktop_config.json` file: ```json { "mcpServers": { "resemble-ai": { "sseUrl": "http://localhost:8083/sse" } } } ``` ### StdIO Transport Connection Create a `claude_desktop_config.json` file: ```json { "mcpServers": { "resemble-ai": { "command": "python", "args": ["-m", "src.cli", "--implementation", "stdio"], "env": { "RESEMBLE_API_KEY": "your_api_key_here" }, "disabled": false, "autoApprove": [] } } } ``` ## Connecting to Cursor ### SSE Transport Connection 1. Go to Settings → AI → MCP Servers 2. Click "Add Server" 3. Select "SSE" as the connection type 4. Set the URL to: `http://localhost:8083/sse` ### StdIO Transport Connection 1. Go to Settings → AI → MCP Servers 2. Click "Add Server" 3. Select "Subprocess" as the connection type 4. Set the command to: `python -m src.cli --implementation stdio` 5. Optionally add environment variables: - `RESEMBLE_API_KEY`: Your Resemble AI API key ## Available Tools ### `list_voices` Lists available voice models from Resemble AI. ### `generate_tts` Generates voice audio from text. Parameters: - `text`: The text to convert to speech - `voice_id`: ID of the voice to use - `return_type`: How to return the audio: 'file' or 'base64' (optional, default: 'file') - `output_filename`: Filename for the output without extension (optional) ## Implementation Details The project includes several implementations: - `src/resemble_mcp_server.py`: Uses the MCP SDK with SSE transport - `src/resemble_stdio_server.py`: Uses StdIO transport for direct process communication - `src/resemble_http_server.py`: HTTP implementation with SSE (fallback) - `src/resemble_ai_server.py`: Direct API implementation - `src/resemble_ai_sdk_server.py`: Implementation using the official Resemble SDK ## Troubleshooting ### MCP SDK Import Errors If you encounter issues importing the MCP SDK, the server will automatically fall back to the HTTP implementation with SSE transport. ### Connection Issues If Claude or Cursor cannot connect to the server: 1. Check that the server is running 2. Verify the correct URL is configured 3. Check that your API key is valid 4. Look for errors in the server logs ### StdIO vs. SSE Transport - Use **SSE Transport** when you want to run the server separately, or on a different machine - Use **StdIO Transport** when you want Claude/Cursor to manage the server process for you ## Examples Example usage can be found in the `examples/` directory. ## 📁 Repository Structure ``` . ├── src/ # Source code for the server implementations │ ├── resemble_mcp_server.py # MCP SDK implementation (recommended) │ ├── resemble_http_server.py # HTTP API implementation │ ├── resemble_ai_server.py # Direct API implementation │ ├── resemble_ai_sdk_server.py # Resemble SDK implementation │ └── cli.py # CLI tool for running the server ├── tests/ # Test scripts ├── docs/ # Documentation ├── examples/ # Example usage and tools ├── scripts/ # Setup and utility scripts ├── output/ # Generated audio output directory ├── .env.example # Example environment configuration ├── requirements.txt # Python dependencies └── README.md # This file ``` ## 🚀 Quick Setup Two setup scripts are provided to make installation easy: ### Using Conda (Recommended) ```bash # Make the script executable chmod +x scripts/setup_environment.sh # Run the setup script ./scripts/setup_environment.sh ``` ### Using Python venv ```bash # Make the script executable chmod +x scripts/setup_venv.sh # Run the setup script ./scripts/setup_venv.sh ``` Either script will: 1. Create a Python 3.10+ environment 2. Install all required dependencies 3. Set up a template .env file 4. Create the output directory for audio files ## Manual Installation If you prefer to set up manually: 1. Create a Python 3.10+ environment: ```bash # Using conda conda create -n resemble_mcp python=3.10 conda activate resemble_mcp # OR using venv (with Python 3.10+ already installed) python3.10 -m venv venv source venv/bin/activate ``` 2. Install dependencies: ```bash pip install uvicorn fastapi python-dotenv requests pydantic httpx sse-starlette pip install git+https://github.com/modelcontextprotocol/python-sdk.git ``` 3. Set up your environment variables: ```bash cp .env.example .env ``` Edit the `.env` file and add your Resemble AI API key: ``` RESEMBLE_API_KEY=your_api_key_here ``` Optional: Customize audio output settings ``` OUTPUT_DIR=./output AUDIO_FORMAT=mp3 ``` 4. Create output directory: ```bash mkdir -p output ``` ## 🚀 Running the Server You can run the server using our new CLI tool which supports all implementations: ```bash # Activate your environment if not already activated conda activate resemble_mcp # OR source venv/bin/activate # Run the MCP SDK implementation (recommended) python -m src.cli --implementation mcp --port 8083 # Other implementations: # HTTP API implementation python -m src.cli --implementation http --port 8083 # Direct API implementation python -m src.cli --implementation direct --port 8083 # Resemble SDK implementation python -m src.cli --implementation sdk --port 8083 ``` ## 🔌 Integrating with Cursor AI Cursor can interact with the Resemble AI voice generation server through the SSE interface: 1. In Cursor, go to Settings → AI → MCP Servers 2. Click "Add Server" and enter the SSE URL: `http://localhost:8083/sse` (adjust port if needed) 3. Save the configuration ## 🔌 Integrating with Claude Desktop 1. Configure the MCP Server in Claude Desktop settings: ```json { "mcpServers": { "resemble-ai": { "command": "python", "args": ["-m", "src.cli", "--implementation", "mcp"], "env": { "RESEMBLE_API_KEY": "your_api_key_here" }, "disabled": false, "autoApprove": [] } } } ``` ## 🛠️ Tool Documentation ### list_voices Lists all available voice models from Resemble AI. **Parameters:** None **Returns:** - `voices`: List of available voice models with IDs, names, genders, languages, accents, and descriptions ### generate_tts Generates voice audio from text. **Parameters:** - `text` (string, required): The text to convert to speech - `voice_id` (string, required): The ID of the voice to use - `return_type` (string, optional): How to return the audio: 'file' or 'base64' (default: 'file') - `output_filename` (string, optional): Filename for the output without extension (default: auto-generated name) **Returns:** - `success` (boolean): Whether the operation was successful - `message` (string): Status message - `audio_data` (string, optional): Base64-encoded audio data (if return_type is 'base64') - `file_path` (string, optional): Path to the saved audio file (if return_type is 'file') ## 💬 Example Prompts Once connected to Cursor or Claude Desktop, you can use prompts like: **Listing available voices:** ``` List all available voice models from Resemble AI. ``` **Generating voice audio:** ``` Generate audio of the text "Hello, this is a test of the Resemble AI voice generation system" using a male English voice. ``` ## ⚠️ Troubleshooting - **Python Version Issues**: The MCP package requires Python 3.10 or newer. Use the provided setup scripts to create the correct environment. - **API Connection Issues**: Make sure you're using the correct API endpoint. The Resemble AI API endpoint is `https://app.resemble.ai/api/v2/`. - **Authentication Errors**: Verify your API key is correct and not expired. - **Missing Projects**: The API requires at least one project in your Resemble account. Create a project through the Resemble AI dashboard if needed. - **Cursor SSE Connection Errors**: If Cursor fails to connect via SSE, make sure: - The server is running on the specified port - You're using the correct `/sse` endpoint - No firewall is blocking the connection - Try restarting both the server and Cursor ## 📚 Additional Documentation For more detailed documentation, see the files in the `docs/` directory. ## 📄 License MIT