Karya
by Dashy-E
README.md
# Karya
**An AI back-office agent that makes phone calls, and the MCP server that gives it hands.**
You type this:
```
you › call Arpit Dash and keep talking until you have his notice period and expected CTC
```
An AI voice agent phones him, holds a real conversation, and you get this back:
```json
{
"still_interested": true,
"notice_period_days": 60,
"expected_ctc_lpa": 26,
"current_location": "Bengaluru"
}
```
Everything runs end to end **with no API keys and no spend** — calls are simulated by a
scripted provider until you flip one environment variable.
```bash
pnpm install && pnpm web # → http://localhost:4000
```
That opens the **demo console**: a browser UI where you type an instruction, watch every
MCP tool call as it fires, and watch a call progress through `dialing → ringing →
talking → done` with the requested fields filling in live. See
[docs/demo.md](docs/demo.md) for a five-minute presenting script.
---
## Contents
- [What this is](#what-this-is)
- [Quick start](#quick-start)
- [The demo console](#the-demo-console)
- [Architecture](#architecture)
- [Project structure](#project-structure)
- [Adding a tool](#adding-a-tool)
- [Adding a provider](#adding-a-provider)
- [Tools, resources and prompts](#tools-resources-and-prompts)
- [Configuration](#configuration)
- [Going live](#going-live)
- [Cost](#cost)
- [Compliance](#compliance)
- [Development](#development)
- [Deployment](#deployment)
---
## What this is
Two packages that are deliberately kept apart:
| Package | What it is | Who uses it |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| **`@karya/mcp-server`** | The capability layer — voice, SMS, WhatsApp, email, CRM, tasks — exposed as MCP tools, resources and prompts. Knows nothing about humans. | Any MCP client: Claude Desktop, your own orchestrator, the ElevenLabs voice agent |
| **`@karya/agent`** | A Gemini-powered CLI that an operator talks to in English. Knows nothing about Twilio. | You, at a terminal |
| **`@karya/web`** | A browser console over the same agent — chat, live call panel, tool trace. The demo surface. | You, presenting |
That separation is the point. The server is the durable asset — drop it into a larger
agent later and every capability comes along. The CLI and the web console are just two
clients of it, and there is nothing special about either.
### The idea that makes this work
There are **two different agents**, and conflating them is the usual mistake:
- the **orchestrator** (Gemini, text) plans and talks to _you_;
- the **voice agent** (ElevenLabs, audio) talks to the _candidate_, live, on the phone.
We do not build the second one's audio loop. ElevenLabs Agents already owns ASR → LLM →
TTS → turn-taking with a native Twilio integration, and — crucially — it can connect to a
**custom MCP server over HTTP**. So the same server serves both: your orchestrator over
stdio, the voice agent over HTTP for its in-call tools.
```
Operator (human, English)
│
▼
┌───────────────────┐
│ @karya/agent │ Gemini Flash · text · plans and reports
└─────────┬─────────┘
│ MCP (stdio)
▼
┌─────────────────────────────────────────────┐
│ @karya/mcp-server │
│ tools · services · provider ports │
└──┬────────────┬─────────────┬───────────────┘
│ │ │
▼ ▼ ▼
Twilio SendGrid ElevenLabs Agents ──── native ──► Twilio Voice
SMS Email (voice agent) │
▲ ▼
MCP (HTTP) ─────┘ Candidate
in-call tools + post-call webhook ──► back into the server
```
Note the loop at the bottom. The voice agent calls back into this same server mid-call
(`save_candidate_field`, `lookup_contact`), and when the call ends ElevenLabs POSTs the
transcript and its extracted fields to our webhook. **That is where structured output
comes from** — not from regex over a transcript.
---
## Quick start
Requires **Node ≥ 20.11** and **pnpm**.
```bash
corepack enable pnpm # or: npm install -g pnpm
pnpm install
pnpm test # 94 tests, no credentials needed
```
### The fastest way to see it
```bash
pnpm web # → http://localhost:4000
```
The browser console. Type `call Arpit Dash and find out his notice period`, and watch the
tool calls and the call lifecycle. No keys needed. A presenting script is in
[docs/demo.md](docs/demo.md).
### The same flow, in a terminal
```bash
pnpm example:client
```
This spawns the server, lists every tool/resource/prompt, looks up a contact, places a
(simulated) call, polls it to completion, prints the structured result, and reads the
transcript resource. No keys, no spend.
### Talk to the agent
Get a free Gemini key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey):
```bash
cp .env.example .env # then set GEMINI_API_KEY
pnpm agent
```
```
you › look up everyone tagged candidate
you › call Arpit Dash and find out his notice period and what he expects to be paid
you › now email him a summary of that conversation
```
You will see each tool call as it happens:
```
→ lookup_contact query="Arpit Dash"
✓ Found Arpit Dash (cnt_demo_arpit). Phone: +919876543210.
→ make_phone_call to="cnt_demo_arpit" objective="Screen the candidate…"
✓ Calling Arpit Dash (task tsk_01KZ6P…). Collecting: notice_period_days, expected_ctc_lpa.
→ get_call_result task_id="tsk_01KZ6P…"
✓ Call completed in 13s. Collected: notice_period_days=60, expected_ctc_lpa=26.
```
### Use it from Claude Desktop
```bash
pnpm build
```
Then copy the `karya` block from [`examples/claude-desktop-config.json`](examples/claude-desktop-config.json)
into your Claude Desktop config and restart.
---
## The demo console
```bash
pnpm web # → http://localhost:4000
```
```
┌──────────────────────────────────────────────┬───────────────────────────┐
│ Karya MOCK MODE gemini-flash-latest 14 tools 09:00–21:00 ● live │
├──────────────────────────────────────────────┼───────────────────────────┤
│ │ LIVE CALL talking │
│ Operator │ ┌─────────────────────┐ │
│ call Arpit Dash and find out his notice │ │ AD Arpit Dash │ │
│ period and expected CTC │ │ +919876543210 │ │
│ │ │ ▓▓▓▓▓▓▓░░░░░ 3/4 │ │
│ │ → lookup_contact query="Arpit Dash" │ │ notice period 60 │ │
│ │ ✓ Found Arpit Dash (cnt_demo_arpit) │ │ expected ctc 26 │ │
│ │ → make_phone_call to="cnt_demo_arpit" │ │ location Bengaluru │ │
│ │ ✓ Calling Arpit Dash (task tsk_01KZ…) │ │ relocate waiting… │ │
│ │ └─────────────────────┘ │
│ Karya │ TOOL CALLS │
│ Calling Arpit Dash. I'll collect: │ CONTACTS │
│ notice_period_days, expected_ctc_lpa. │ RECENT TASKS │
├──────────────────────────────────────────────┤ │
│ [ type an instruction… ] ➤ │ │
└──────────────────────────────────────────────┴───────────────────────────┘
```
**What it shows**
- **Every tool call, inline** — name, arguments, and result, as they happen. The chat is
not a black box.
- **The call lifecycle** as a live stage rail, because the asynchrony is the design.
- **Fields filling in one by one**, typed. A field the call failed to obtain shows as
`not obtained` in amber — never blank, never invented.
- **Contacts with a `blocked` flag**, so a compliance refusal is visible before you
trigger it.
**How it works.** The browser holds one SSE connection. Behind it, a small Express server
owns a **real MCP client** wired to a **real MCP server** over an in-memory transport —
every action is a genuine `tools/call` through the full middleware pipeline. The session
is shared, so a laptop and a projector can watch the same call.
**Two planners.** With `GEMINI_API_KEY` set you get the real LLM agent. Without one it
falls back to a deterministic keyword planner that drives the same tools — so a missing
key or an exhausted free-tier quota degrades the demo instead of ending it. The UI always
says which is active.
Clicking a contact drafts an instruction rather than sending one; an outbound call should
never be one stray click away.
---
## Architecture
Four layers. **Dependencies point strictly inward** — business logic never sees a vendor.
```
┌──────────────────────────────────────────┐
│ server/ transport, McpServer wiring │ ← protocol edge
├──────────────────────────────────────────┤
│ tools/ resources/ prompts/ │ ← MCP surface (thin)
│ + core/ registry, middleware │
├──────────────────────────────────────────┤
│ services/ business logic (use cases) │ ← knows only ports
├──────────────────────────────────────────┤
│ providers/ ports (interfaces) │
│ adapters/ memory | elevenlabs | twilio │ ← swappable I/O
└──────────────────────────────────────────┘
cross-cutting: config, logger, errors, store, utils
```
### Design decisions worth knowing
**A phone call is a long-running job, not a request.** `make_phone_call` returns a
`task_id` in milliseconds. The call then runs for minutes. Anything that awaited the
outcome would hold an MCP request open for ten minutes, break every client timeout, and
lose the result entirely on a reconnect.
**Tools are values, not registration calls.** Each tool exports a `ToolDefinition`
object; the registry — not the tool — talks to the MCP SDK. So a tool is unit-testable
by calling `execute(input, fakeCtx)` with no server, no transport and no SDK mocking.
**`execute(input, ctx)` receives its dependencies.** That _is_ the dependency injection.
No DI container, no decorators, no reflection anywhere in this codebase.
**A middleware pipeline wraps every invocation** — logging/timing → authorization →
timeout → validation. Cross-cutting concerns are applied uniformly instead of being
re-typed, slightly differently, in each tool.
**Two audiences, enforced.** The ElevenLabs voice agent is mid-conversation with a
candidate; it must never see `send_email`. `exposeTo` hides out-of-audience tools from
`tools/list` _and_ refuses them at invocation, because "not listed" is not "not callable".
**Missing data is reported, never inferred.** Every call result carries
`missing_fields`. Returning nulls invites a model to fill them in; naming the gap
explicitly is what stops a fabricated salary reaching an operator.
**Compliance is code, not documentation.** TRAI calling hours, DND and consent are
enforced by `ComplianceService` on every outbound path, and refusals are marked
"do not retry" so a model does not turn one violation into ten.
---
## Project structure
```
karya/
├─ packages/
│ ├─ shared/ # domain schemas used by BOTH packages
│ │ └─ src/ # primitives · contact · call · task · messaging
│ │
│ ├─ mcp-server/src/
│ │ ├─ index.ts # entrypoint + public API
│ │ ├─ server/ # McpServer wiring, transports, bootstrap, shutdown
│ │ │ └─ transports/ # stdio.ts · http.ts
│ │ ├─ core/ # ToolDefinition, registries, middleware ← the contracts
│ │ ├─ tools/ # one folder per tool
│ │ │ ├─ voice/ messaging/ crm/ scheduling/ tasks/ utilities/
│ │ │ ├─ incall/ # voice-agent-only tools
│ │ │ └─ index.ts # the single tool manifest
│ │ ├─ resources/ # transcripts, KB, policies, contacts, docs
│ │ ├─ prompts/ # recruiter screening, support, booking, …
│ │ ├─ services/ # use cases; depend on ports only
│ │ ├─ providers/
│ │ │ ├─ ports/ # Voice · Sms · WhatsApp · Email · Crm · Knowledge
│ │ │ ├─ adapters/memory/ # ★ full simulator — this is why it's free
│ │ │ ├─ adapters/elevenlabs|twilio|sendgrid/
│ │ │ └─ factory.ts # the ONLY file mapping config → concrete class
│ │ ├─ webhooks/ # ElevenLabs post-call receiver (HMAC verified)
│ │ ├─ store/ # JSON-file repositories, no native deps
│ │ └─ config/ logger/ errors/ utils/
│ │
│ ├─ agent/src/
│ │ ├─ cli.ts # the REPL
│ │ ├─ loop.ts # Gemini function-calling loop
│ │ ├─ mcp-client.ts # MCP tools → Gemini declarations; ToolHost interface
│ │ ├─ schema-bridge.ts # JSON Schema → Gemini schema ← subtle, well tested
│ │ └─ system-prompt.ts
│ │
│ └─ web/ # DEMO CONSOLE
│ ├─ src/
│ │ ├─ session.ts # in-process MCP client + live call watcher
│ │ ├─ server.ts # Express: SSE stream, chat API, static hosting
│ │ ├─ events.ts # the browser event contract
│ │ └─ planner/ # gemini.ts | scripted.ts (no-API-key fallback)
│ └─ public/ # index.html · styles.css · app.js (no build step)
│
├─ tests/ unit · tools · validation · integration
├─ examples/ runnable MCP client · Claude Desktop config
└─ docs/ india-compliance · going-live · architecture
```
---
## Adding a tool
Three small files and one import line.
**1.** `src/tools/crm/delete-contact/schema.ts`
```ts
import { z } from 'zod';
export const DeleteContactInput = z.object({
contact_id: z.string().describe('Id of the contact to delete.'),
});
export const DeleteContactOutput = z.object({
deleted: z.boolean(),
contact_id: z.string(),
});
```
**2.** `src/tools/crm/delete-contact/execute.ts`
```ts
import type { ToolContext } from '../../../core/tool.js';
import type { z } from 'zod';
import type { DeleteContactInput, DeleteContactOutput } from './schema.js';
export const execute = async (
input: z.infer<typeof DeleteContactInput>,
ctx: ToolContext,
): Promise<z.infer<typeof DeleteContactOutput>> => {
await ctx.services.contacts.delete(input.contact_id);
return { deleted: true, contact_id: input.contact_id };
};
```
**3.** `src/tools/crm/delete-contact/index.ts`
```ts
import { defineTool } from '../../../core/tool.js';
import { execute } from './execute.js';
import { DeleteContactInput, DeleteContactOutput } from './schema.js';
export const deleteContactTool = defineTool({
name: 'delete_contact',
title: 'Delete a contact',
category: 'crm',
exposeTo: ['operator'],
description: 'Permanently remove a contact. Use when someone asks to be erased.',
inputSchema: DeleteContactInput,
outputSchema: DeleteContactOutput,
annotations: { destructiveHint: true, idempotentHint: true },
execute,
});
```
**4.** Add it to `src/tools/index.ts`:
```ts
import { deleteContactTool } from './crm/delete-contact/index.js';
export const allTools = [/* … */ deleteContactTool];
```
That is all. Validation, request ids, timing logs, timeouts, audience checks and error
shaping are supplied by the pipeline — write business logic only, and throw any
`KaryaError` freely.
> **Why the manifest instead of globbing the directory?** Globbing appears to remove
> that one line, but it breaks bundling and tree-shaking, defeats TypeScript (a broken
> tool becomes a runtime surprise rather than a compile error), and makes the tool set
> unknowable without running the app. The single failure it prevents — forgetting the
> line — is covered by `tests/tools/registry.test.ts`, which walks the tree and fails if
> a tool folder is missing from the array.
### Writing a good description
Tool descriptions are read by a _language model_, and most tool misuse is a description
problem rather than a model problem. State what it does, when to use it, and — most
valuable — **when not to**. Compare `make_phone_call`:
> Do NOT use this for a text message (use send_sms), and do not call it a second time
> for the same person while an earlier call is still in progress.
---
## Adding a provider
One adapter file, one line in the factory, credentials in the config schema.
**1.** Implement the port (`src/providers/ports/voice.ts`):
```ts
export class ExotelVoiceProvider implements VoiceProvider {
readonly name = 'exotel';
async startCall(request: StartCallRequest): Promise<StartCallResponse> {
/* … */
}
async getCall(conversationId: string): Promise<CallResult | null> {
/* … */
}
async endCall(conversationId: string): Promise<void> {
/* … */
}
}
```
**2.** Register it in `src/providers/factory.ts`:
```ts
case 'exotel':
return new ExotelVoiceProvider({ apiKey: config.exotel.apiKey, logger });
```
**3.** Add its credentials and requirements to `src/config/schema.ts`.
No service and no tool changes — nothing in `services/` or `tools/` has ever heard of
Twilio. This is exactly the path to take if you need an Indian caller ID
(see [compliance](#compliance)).
---
## Tools, resources and prompts
### Tools (15)
| Tool | Category | Audience | Notes |
| ------------------------ | ---------- | --------------- | -------------------------------- |
| `make_phone_call` | voice | operator | Async — returns a `task_id` |
| `get_call_result` | voice | operator | Poll until `is_final` |
| `end_call` | voice | operator | Hang up early |
| `send_sms` | messaging | operator | |
| `send_whatsapp` | messaging | operator | Higher read rates in India |
| `send_email` | messaging | operator | |
| `lookup_contact` | crm | both | Call this before dialling a name |
| `create_contact` | crm | operator | |
| `update_contact` | crm | operator | Consent, opt-out, notes |
| `schedule_callback` | scheduling | both | Records intent; does not dial |
| `get_task_status` | tasks | operator | Any channel |
| `list_tasks` | tasks | operator | |
| `search_knowledge_base` | utilities | both | |
| `summarize_conversation` | utilities | operator | Returns material; you summarise |
| `save_candidate_field` | incall | **voice agent** | Incremental save during a call |
### Resources
| URI | What |
| ---------------------------------- | ------------------------------------------------------ |
| `karya://docs/tools` | Live capability reference, generated from the registry |
| `karya://policies/operating-rules` | Compliance rules, rendered from live config |
| `karya://kb/{articleId}` | Knowledge base article |
| `karya://transcripts/{taskId}` | Full transcript and outcome of a call |
| `karya://contacts/{contactId}` | Contact record and preferences |
### Prompts
`recruitment_screening` · `customer_support_callback` · `appointment_booking` ·
`sales_outreach` · `payment_reminder` · `follow_up_call`
Run one from the CLI:
```
/run recruitment_screening candidate="Arpit Dash" role="Senior Backend Engineer"
```
---
## Configuration
Every variable is documented in [`.env.example`](.env.example). Configuration is
validated at startup and the server **refuses to start** if anything is wrong — naming
_every_ problem at once rather than one per restart:
```
Invalid Karya configuration — 3 problem(s) found:
• ELEVENLABS_API_KEY: ELEVENLABS_API_KEY is required when the voice provider is elevenlabs.
• TWILIO_ACCOUNT_SID: TWILIO_ACCOUNT_SID is required when a Twilio provider is enabled.
• KARYA_HTTP_AUTH_TOKEN: … An unauthenticated MCP endpoint exposes your phone and
email accounts to anyone who finds the URL.
```
The single most important variable:
```bash
KARYA_PROVIDER_MODE=mock # in-memory adapters, no keys, no spend (default)
KARYA_PROVIDER_MODE=live # real providers, real people, real money
```
---
## Going live
Full walkthrough in [`docs/going-live.md`](docs/going-live.md). In outline:
1. **Twilio** — buy a **US** number (see the India note below). Note the Account SID and
Auth Token.
2. **ElevenLabs** — create an agent, then add your Twilio number under _Phone Numbers_
(paste SID + auth token; it auto-configures the webhooks). Note the agent id and
phone number id.
3. **Expose your server** — `cloudflared tunnel --url http://localhost:3000` (free).
4. **Post-call webhook** — point ElevenLabs at
`https://<tunnel>/webhooks/elevenlabs` and copy the signing secret into
`ELEVENLABS_WEBHOOK_SECRET`. **Without this, results never arrive and calls stay
"running" forever.**
5. **In-call tools** — add a custom MCP server in ElevenLabs pointing at
`https://<tunnel>/mcp/voice-agent`, with `Authorization: Bearer <KARYA_HTTP_AUTH_TOKEN>`.
Use **fine-grained approval** and auto-approve only the read-safe tools.
6. Set `KARYA_PROVIDER_MODE=live` and start with `KARYA_TRANSPORT=http`.
---
## Cost
| Layer | Choice | Cost |
| ------------------------------- | --------------------------------------------- | --------------------------------- |
| Orchestrator LLM | Gemini Flash, free tier | **$0** (10 req/min, 250/day) |
| Voice agent LLM | Point ElevenLabs at Gemini via Custom LLM | avoids the platform LLM markup |
| Voice (ASR + TTS + turn-taking) | ElevenLabs Agents | **the dominant cost**, per-minute |
| Telephony | Twilio US → India | ~$1.15/mo number + ~$0.0496/min |
| Email | SendGrid | free tier covers low volume |
| Hosting | stdio locally; Cloudflare Tunnel for webhooks | $0 |
**The biggest lever is not the vendors — it is mock mode.** The full system, including a
scripted phone conversation, runs on in-memory adapters. Develop and test for free; flip
to `live` only for real calls. Every call result also reports its own estimated cost, and
`KARYA_MAX_CALL_DURATION_SECONDS` caps the exposure of any single call.
> ElevenLabs' exact per-minute rate and free-tier minute allowance are not reproduced
> here because they change; check their current pricing before you budget.
---
## Compliance
**Read [`docs/india-compliance.md`](docs/india-compliance.md) before calling Indian
numbers.** The essentials:
⚠️ **Twilio has not supported outbound calls from Indian (+91) numbers since 1 August
2024**, and its India guidelines state that calls _to_ India may only be placed from
non-Indian numbers. Karya therefore dials Indian candidates from a **US number**, so the
recipient sees a foreign caller ID and pickup rates suffer. If a +91 caller ID is a
business requirement, you need an Indian provider (Exotel, Plivo India, Knowlarity) —
which is a one-adapter change, see [adding a provider](#adding-a-provider).
Enforced in code, on every outbound path:
- **Calling hours** — 09:00–21:00 in the _recipient's_ time zone. Outside it, calls are
refused, not queued.
- **Consent** — contacts with no lawful basis are blocked. For recruitment the
defensible basis is `applied`.
- **Opt-out** — `doNotContact` blocks every channel, with no override.
- **Duration cap** — clamped, not honoured, above the configured ceiling.
Not enforced in code, and your responsibility: DLT registration, 140/1600-series
numbering, and DND scrubbing.
---
## Development
```bash
pnpm typecheck # tsc -b, strict, project references
pnpm lint # eslint, zero warnings
pnpm format # prettier
pnpm test # vitest — 94 tests
pnpm test:watch
pnpm test:coverage
pnpm build # tsup → dist/
pnpm verify # everything above, in order
```
Useful entry points:
```bash
pnpm web # the browser demo console (builds deps first)
pnpm web:only # same, skipping the dependency build
pnpm mcp # server on stdio
pnpm mcp:http # server on HTTP (needs KARYA_HTTP_AUTH_TOKEN)
pnpm agent # the CLI
pnpm example:client # the runnable example
```
### Testing philosophy
Tests run entirely against in-memory adapters with a **frozen clock**. That combination
is what makes calling-window behaviour testable at all — otherwise it is a test that
passes until 21:00 and then starts failing nightly.
The test harness (`tests/helpers/harness.ts`) stands up the whole system in one call and
invokes tools through the _real_ middleware pipeline and registry — not a parallel test
path that can drift from what ships.
```ts
const harness = await createHarness({ now: '2026-08-04T18:00:00Z' }); // 23:30 IST
const error = await harness.callErr('make_phone_call', {/* … */});
expect(error.code).toBe('COMPLIANCE_ERROR');
```
---
## Deployment
**stdio** (local, Claude Desktop): `pnpm build`, then point the client at
`packages/mcp-server/dist/index.js`.
**HTTP** (hosted, and required for the ElevenLabs voice agent):
```bash
KARYA_TRANSPORT=http KARYA_HTTP_AUTH_TOKEN=$(openssl rand -hex 32) pnpm mcp:http
```
| Endpoint | Purpose |
| --------------------------- | --------------------------------------------------- |
| `GET /health` | Liveness. Unauthenticated, deliberately detail-free |
| `POST /mcp` | MCP for the operator audience (bearer auth) |
| `POST /mcp/voice-agent` | MCP for the ElevenLabs voice agent (bearer auth) |
| `POST /webhooks/elevenlabs` | Post-call results (HMAC verified) |
Sessions are stateless — a fresh `McpServer` per request, with services and the store
shared and long-lived. There is no session table to leak, expire, or lose on restart.
A `Dockerfile` is included. Mount a volume at `KARYA_DATA_DIR` if you want tasks and
transcripts to survive a restart.
---
## License
MIT
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues