# elevenlabs-call-mcp

A remote **MCP server** that lets [Poke](https://poke.com) place real outbound phone calls through **ElevenLabs Conversational AI** (over Twilio). Tell Poke *"Call the dentist and book a cleaning next week"* and an AI voice agent makes the call and reports back.

Runs on **Cloudflare Workers** (TypeScript). ElevenLabs + Twilio own the real-time voice streaming; this server is the control plane that starts calls and relays results.

## Architecture

```
Poke  ──MCP (HTTP/JSON-RPC)──▶  Worker  ──REST──▶  ElevenLabs ConvAI ──▶ Twilio ──▶ ☎ callee
  ▲                              │  ▲
  └────push notification─────────┘  └──post-call webhook──┘
        (Poke inbound API)
```

- `place_call` provisions a per-call ElevenLabs agent tailored to the objective, triggers `POST /v1/convai/twilio/outbound-call`, and returns a `call_id` **immediately** (calls run for minutes — the tool never blocks).
- Completion is observed two ways: the `get_call_status` tool polls `GET /v1/convai/conversations/{id}`, **and** an ElevenLabs post-call **webhook** pushes a summary back to the user via the Poke inbound API. Both funnel through one idempotent finalize routine (notify once).

### MCP tools

| Tool | Purpose |
|------|---------|
| `place_call` | Start an outbound call pursuing a natural-language `objective` to a `to_number` (E.164), with optional `caller_context`. |
| `get_call_status` | Status + transcript + outcome summary for a `call_id`. |
| `list_calls` | The user's recent calls, most recent first. |
| `cancel_call` | Best-effort cancel of an in-flight call. |

## Project layout

```
src/
  index.ts        HTTP router: /mcp, /webhook, /health
  mcp.ts          MCP JSON-RPC handler (initialize, tools/list, tools/call)
  tools.ts        The four tools + input validation (zod)
  elevenlabs.ts   Thin ElevenLabs ConvAI client
  poke.ts         Poke inbound-API client (push-back)
  finalize.ts     Idempotent reconcile/finalize (notify-once, agent cleanup)
  webhook.ts      Post-call webhook verification + processing
  store.ts        KV-backed CallStore (per-user scoping, rate limit, indexes)
  summarize.ts    Transcript → short outcome summary
  auth.ts / env.ts / types.ts
test/             vitest unit tests
```

## Prerequisites / accounts

- **ElevenLabs** account + API key, with a Conversational AI **phone number linked to Twilio** (gives an `agent_phone_number_id`).
- **Twilio** account + a voice-capable number imported into ElevenLabs.
- **Poke** account + inbound API key (for push-back notifications).
- **Cloudflare** account (Workers + KV).

## Setup

```bash
npm install

# 1. Create the KV namespace and paste the id into wrangler.toml
wrangler kv namespace create CALLS

# 2. Set secrets (never committed)
wrangler secret put ELEVENLABS_API_KEY        # ElevenLabs xi-api-key
wrangler secret put TWILIO_PHONE_NUMBER_ID    # ElevenLabs agent_phone_number_id (Twilio-linked)
wrangler secret put POKE_API_KEY              # Poke inbound API key
wrangler secret put WEBHOOK_SECRET            # shared secret for the post-call webhook HMAC
wrangler secret put MCP_BEARER_TOKEN          # token Poke must present to call /mcp

# 3. (optional) tune non-secret vars in wrangler.toml
#    DESTINATION_ALLOWLIST="+14155551234,+442071234567"   # empty = allow any (not recommended)
#    RATE_LIMIT_PER_HOUR="10"
#    AGENT_LLM="claude-haiku-4-5"            # any ElevenLabs-supported LLM enum value
#    TTS_MODEL_ID="eleven_v3_conversational" # realtime v3; plain "eleven_v3" is gated (see table below)
#    TTS_VOICE_ID=""                         # specific voice, or blank for the default
#    AGENT_LANGUAGE="en"

# 4. Deploy
npm run deploy
```

### Configure the ElevenLabs post-call webhook

**Post-call webhooks are configured in the ElevenLabs dashboard at the workspace level — they are NOT registered by this server's agent-creation code.** Without this, calls still run but completion notifications never fire.

1. ElevenLabs dashboard → **Settings → Webhooks** → create a webhook with URL:
   ```
   https://<your-worker>.workers.dev/webhook
   ```
2. Copy the **signing secret** ElevenLabs generates for that webhook and set it as your Worker secret:
   ```bash
   wrangler secret put WEBHOOK_SECRET   # paste the ElevenLabs signing secret
   ```
   The Worker verifies the `ElevenLabs-Signature` header (`t=<ts>,v0=<hmac-sha256>` of `"<ts>.<body>"`) against this secret, so the two **must match** or webhooks are rejected with 401.
3. In **Agents → Settings**, enable the **post-call transcription webhook** (and, if your workspace uses per-agent overrides, enable it on the agent too).

> Because agents are created per-call by this server, prefer enabling the post-call webhook **workspace-wide** so every generated agent inherits it.

### Agent model selection

Each call provisions a fresh agent using the model settings from `wrangler.toml` `[vars]`:

| Var | Default | Notes |
|-----|---------|-------|
| `AGENT_LLM` | `claude-haiku-4-5` | Any [supported LLM](https://elevenlabs.io/docs/eleven-agents/customization/llm) enum value. |
| `TTS_MODEL_ID` | `eleven_v3_conversational` | Realtime v3 voice. **Heads up:** the plain `eleven_v3` id is the *gated "expressive"* model and returns `expressive_tts_not_allowed` on accounts without that feature — use the `_conversational` variant for live calls (or fall back to `eleven_turbo_v2_5` / `eleven_flash_v2_5`). |
| `TTS_VOICE_ID` | _(default)_ | A specific voice id, or blank for the workspace default. |
| `AGENT_LANGUAGE` | `en` | Agent language. |

These map to `conversation_config.agent.prompt.llm`, `conversation_config.tts.model_id` / `voice_id`, and `conversation_config.agent.language`.

### Register the server in Poke

Either the web app or CLI:

```bash
# Web: poke.com/integrations/new  → name + URL https://<your-worker>.workers.dev/mcp
# CLI:
npx poke@latest mcp add https://<your-worker>.workers.dev/mcp -n "Phone Calls" -k "<MCP_BEARER_TOKEN>"
```

Poke sends `Authorization: Bearer <token>` and `X-Poke-User-Id` on every request; the Worker enforces both (bearer = auth, user id = scoping).

## Local development

```bash
npm run dev                                   # wrangler dev on http://localhost:8787
npx poke@latest tunnel http://localhost:8787/mcp -n "Phone Calls (dev)"
```

## Testing

```bash
npm test          # vitest unit tests (store scoping, rate limit, finalize idempotency,
                  # status mapping, summary, webhook signature, MCP dispatch)
npm run typecheck
```

## Logs

Workers observability is enabled (`[observability]` in `wrangler.toml`), so invocation logs are retained and browsable in the Cloudflare dashboard (Workers → `elevenlabs-call-mcp` → Logs). For a live stream:

```bash
npx wrangler tail elevenlabs-call-mcp --format pretty
```

Useful log lines: `webhook: finalized <call_id> -> done` (full loop succeeded), `place_call failed for <id>: …` (raw upstream error), and `webhook rejected: …` (signature verification failed — usually a `WEBHOOK_SECRET` mismatch).

## Safety notes

This server **places real phone calls that cost money and reach real people.** Guardrails:
- Bearer-token auth on the MCP endpoint; HMAC-verified webhook.
- Per-user rolling-hour rate limit (`RATE_LIMIT_PER_HOUR`).
- Optional destination allowlist (`DESTINATION_ALLOWLIST`) — strongly recommended in production.
- Per-user call isolation via `X-Poke-User-Id`; cross-user reads return not-found.
- Secrets are never echoed in tool output, logs, or errors.

## Status & known limitations

Verified working end-to-end: `place_call` → per-call agent (Claude Haiku 4.5 + `eleven_v3_conversational`) → Twilio outbound → ElevenLabs processing → signed post-call webhook → `/webhook` (HMAC verified) → push to Poke.

Remaining caveats:
- **Poke push**: the inbound-API call returns 2xx (see `src/poke.ts` for the exact endpoint/payload); user-visible delivery still worth a manual confirm if you change the payload.
- **`cancel_call` is best-effort** — ElevenLabs exposes no first-class outbound-call kill switch; the server marks the record cancelled and tears down the agent, but an already-connected call may take a moment to drop.
- **TTS model is account-gated** — `eleven_v3_conversational` works here; plain `eleven_v3` requires the Expressive TTS feature.
- **Rate limit is a soft limit** — the per-user KV counter isn't transactional, so it can be slightly exceeded under bursty concurrency.