# Phony

**AI-Powered Voice Call & SMS MCP Server**

A Model Context Protocol (MCP) server that enables Claude and other AI assistants to initiate and manage voice calls and SMS messages using Twilio and OpenAI (GPT-4o Realtime model).

Use this as a base to kick-start your AI-powered voice calling and messaging explorations, save time and develop additional functionality on top of it.

![Demo](./assets/demo.gif)


## Sequence Diagram

```mermaid
sequenceDiagram
    participant AI as AI Assistant (e.g., Claude)
    participant MCP as MCP Server
    participant Twilio as Twilio
    participant Phone as Destination Phone
    participant OpenAI as OpenAI
    
    AI->>MCP: 1) Initiate outbound call request <br>(POST /calls)
    MCP->>Twilio: 2) Place outbound call via Twilio API
    Twilio->>Phone: 3) Ring the destination phone
    Twilio->>MCP: 4) Call status updates & audio callbacks (webhooks)
    MCP->>OpenAI: 5) Forward real-time audio to OpenaAI's realtime model
    OpenAI->>MCP: 6) Return voice stream
    MCP->>Twilio: 7) Send voice stream
    Twilio->>Phone: 8) Forward voice stream
    Note over Phone: Two-way conversation continues <br>until the call ends
```


## Features

### Voice
- Make outbound phone calls via Twilio 📞
- Real-time AI-powered conversations with ElevenLabs Conversational AI or OpenAI GPT-4o Realtime 🎙️
- Real-time language switching during calls 🌐
- In-band DTMF support — AI can autonomously navigate IVR menus
- Pre-built prompts for common calling scenarios (restaurant reservations, etc.) 🍽️

### SMS (1-on-1)
- Send/receive SMS and MMS (media via URLs or base64) 💬
- Persistent MongoDB storage with status tracking (queued → sent → delivered → failed)
- Per-contact **slug proxy system**: incoming messages are mirrored to a configurable list of proxy target numbers (e.g. family members). Those targets can reply with `{slug}: msg` to route back to the external contact.
- Reconciliation loop (5-min poll, 24-hour lookback) — replays anything the webhook missed

### Group MMS (Twilio Conversations API) ⭐
- **True native group MMS threading** on participants' phones — not fan-out
- Autocreate-driven: groups a user starts with Phony in them show up automatically
- `projectedAddress` pattern — Phony joins each group as an avatar, externals join as native SMS
- Slug-based reply routing (`{0101-grp}: msg` from proxy targets posts into the group)
- Proxy targets receive group activity as 1-on-1 SMS when not in the group; skipped when in the group (no native-thread duplication)
- Full participant lifecycle handled (add, remove, message, conversation-removed events)
- Conversation reconciliation and one-shot historical backfill

### Infrastructure
- Public URL support via nginx reverse proxy 🔄
- Secure handling of credentials 🔒
- Docker Compose deployment; MongoDB exposed on loopback for host-side tooling

## Why MCP?

The Model Context Protocol (MCP) bridges the gap between AI assistants and real-world actions. By implementing MCP, this server allows AI models like Claude to:

1. Initiate actual phone calls on behalf of users
2. Process and respond to real-time audio conversations
3. Execute complex tasks requiring voice communication

This open-source implementation provides transparency and customizability, allowing developers to extend functionality while maintaining control over their data and privacy.

## Requirements

- Node.js >= 22
  - If you need to update Node.js, we recommend using `nvm` (Node Version Manager):
    ```bash
    nvm install 22
    nvm use 22
    ```
- Twilio account with API credentials
- OpenAI API key
- Public URL with nginx reverse proxy (or similar) for Twilio webhooks

## Installation

### Manual Installation

1. Clone the repository
   ```bash
   git clone https://github.com/sackio/phony.git
   cd phony
   ```

2. Install dependencies and build
   ```bash
   npm install
   npm run build
   ```

## Configuration

The server requires several environment variables. Copy `.env.example` to `.env` and configure:

```bash
cp .env.example .env
```

### Required Environment Variables

- `PUBLIC_URL`: Your public URL for Twilio callbacks (e.g., `https://your-domain.com`)
- `TWILIO_ACCOUNT_SID`: Your Twilio account SID
- `TWILIO_AUTH_TOKEN`: Your Twilio auth token
- `TWILIO_NUMBER`: Your Twilio number (in E.164 format)
- `OPENAI_API_KEY`: Your OpenAI API key

### MongoDB Security

MongoDB is configured with authentication enabled. Generate a secure password:

```bash
openssl rand -base64 32
```

Then configure in `.env`:

- `MONGODB_USERNAME`: Database username (default: `voicecalls_admin`)
- `MONGODB_PASSWORD`: Randomly generated password (use command above)
- `MONGODB_DATABASE`: Database name (default: `phony`)
- `MONGODB_URI`: Full connection string with authentication

**⚠️ Important**: Never commit `.env` to version control. The `.env.example` file contains placeholder values only.

### Optional Variables

- `RECORD_CALLS`: Set to "true" to record calls (optional)
- `PORT`: Server port (optional, defaults to 3004)
- `API_SECRET`: Webhook security secret (randomly generated if not set)

### Nginx Configuration

You need to configure nginx (or similar reverse proxy) to forward requests to the server. Example nginx location block:

```nginx
# Phony - Twilio webhooks and WebSocket
location /call/ {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 600s;
    proxy_connect_timeout 600s;
    proxy_send_timeout 600s;
    proxy_pass http://localhost:3004;
}
```

**Note**: The `/call/` location should not require authentication as Twilio needs to access these endpoints.

### Claude Desktop Configuration

To use this server with Claude Desktop, add the following to your configuration file:

**macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`

**Windows**: `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "phony": {
      "command": "node",
      "args": ["/path/to/your/phony/dist/start-all.cjs"],
      "env": {
        "PUBLIC_URL": "https://your-domain.com",
        "TWILIO_ACCOUNT_SID": "your_account_sid",
        "TWILIO_AUTH_TOKEN": "your_auth_token",
        "TWILIO_NUMBER": "your_e.164_format_number",
        "OPENAI_API_KEY": "your_openai_api_key"
      }
    }
  }
}
```

After that, restart Claude Desktop to reload the configuration. 
If connected, you should see Voice Call under the 🔨 menu.

## Example Interactions with Claude

Here are some natural ways to interact with the server through Claude:

### Voice calls
1. Simple call:
```
Can you call +1-123-456-7890 and let them know I'll be 15 minutes late for our meeting?
```

2. Restaurant reservation:
```
Please call Delicious Restaurant at +1-123-456-7890 and make a reservation for 4 people tonight at 7:30 PM. Please speak in German.
```

3. Appointment scheduling:
```
Please call Expert Dental NYC (+1-123-456-7899) and reschedule my Monday appointment to next Friday between 4–6pm.
```

### SMS and group MMS

4. Send an SMS:
```
Text +1-978-555-0103 that the door dimensions are 30x80.
```

5. Start a group MMS:
```
Create a group with +1-978-555-0103 (Murilo) and +1-978-555-0104 (Junior) and ask them to confirm they're coming tomorrow morning.
```
The tool (`phony_create_group_conversation`) creates a Twilio Conversation with Phony as the projected address host and sends the first message as a native group MMS to all externals.

6. Post into an existing group:
```
Reply to group {0101-grp}: confirmed, see you at 8am.
```

7. Read the group history:
```
Show me all messages in group {0101-grp} since yesterday.
```

See [`docs/GROUP_MMS_ARCHITECTURE.md`](docs/GROUP_MMS_ARCHITECTURE.md) for the full data flow, Twilio Console configuration, and the participant pattern that makes Group MMS actually render as a single native thread on recipient phones.

## Important Notes

1. **Phone Number Format**: All phone numbers must be in E.164 format (e.g., +11234567890)
2. **Rate Limits**: Be aware of your Twilio and OpenAI account's rate limits and pricing
3. **Voice Conversations**: The AI will handle natural conversations in real-time
4. **Call Duration**: Be mindful of call durations as they affect OpenAI API and Twilio costs
5. **Public Exposure**: Ensure your nginx configuration properly secures endpoints while allowing Twilio webhook access to `/call/` routes

## Troubleshooting

Common error messages and solutions:

1. "Phone number must be in E.164 format"
   - Make sure the phone number starts with "+" and the country code

2. "Invalid credentials"
   - Double-check your TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN. You can copy them from the [Twilio Console](https://console.twilio.com)

3. "OpenAI API error"
   - Verify your OPENAI_API_KEY is correct and has sufficient credits

4. "PUBLIC_URL environment variable is required"
   - Make sure you've set the PUBLIC_URL environment variable to your public domain (e.g., `https://your-domain.com`)
   - Ensure your nginx reverse proxy is properly configured to forward `/call/` requests to the server

5. "OpenAI Realtime does not detect the end of voice input, or is lagging."
   - Sometimes, there might be voice encoding issues between Twilio and the receiver's network operator. Try using a different receiver.

## Contributing

Contributions are welcome! Here are some areas we're looking to improve:

- Implement support for multiple AI models beyond the current implementation
- Add database integration to store conversation history locally and make it accessible for AI context
- Improve latency and response times to enhance call experiences
- Enhance error handling and recovery mechanisms
- Add more pre-built conversation templates for common scenarios
- Implement improved call monitoring and analytics

If you'd like to contribute, please open an issue to discuss your ideas before submitting a pull request.

## Credits

Phony is a fork of the [Voice Call MCP Server](https://github.com/popcornspace/voice-call-mcp-server) originally created by the [Popcorn](https://careers.popcorn.space) team.

We've extended it with:
- SMS messaging capabilities (send, receive, conversation tracking)
- Incoming call handling with proper greeting protocols
- Human-in-the-loop call management with real-time context injection
- React frontend UI for call and SMS management
- Call hold/resume functionality with TTS hold messages
- Enhanced features and comprehensive documentation

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Security

Please do not include any sensitive information (like phone numbers or API credentials) in GitHub issues or pull requests. This server handles sensitive communications; deploy it responsibly and ensure all credentials are kept secure.