Ambient

An Alexa agent that phones a business for you. It says first that it is an AI, never spends past your limit without asking, and books only on a clean yes.

Watch the two-minute film · Try it in one minute, no keys · Try to fool it

The film shows the zero-key demo: the business and the Alexa+ host in it are the simulator, and the voices are text-to-speech. Its source is in launch/film3.

Try it: one minute, no keys

Needs Node 22.13 or later and pnpm (corepack enable installs it). From a clone of this repository:

pnpm install
pnpm demo   # then open http://127.0.0.1:3000/sim

No local setup? Open the repository in GitHub Codespaces. The container installs the project, starts the demo and opens /sim. Each Codespace has its own copy, so two judges never see each other's calls.

You say "Alexa, ask Ambient to call Joe's Plumbing, under 150." Ambient will not book on a price it did not read back, or on a yes with conditions attached. It keeps the yes that booked as proof.

Three moments from the demo:

1. It refuses a false yes. At the read-back, the plumber says "Yes, plus parts." Ambient does not book. It asks for one total price, reads the terms back again, and books on "Yes, that's right."

2. It asks you before it spends more. The plumber quotes $180 against your $150 limit. The call holds, and the Echo Show asks: "Book $180", "Offer $150" or "Decline". The plumber counters at $160, so the Show asks you again.

3. It calls three plumbers at once, in two languages. It books the cheapest quote under your limit and thanks the other two businesses. The cheapest plumber speaks Spanish, so Ambient books in Spanish while you read English on the Show.

Try to fool it yourself. In the simulator, you can play the business. Type any answer to the read-back, in English or Spanish. The same call reducer that runs every call decides, and the panel says whether Ambient would book and why.

The voice line, the Claude brain and the Alexa skill are built and tested against local fakes, but none has run live. See What is real and what is simulated.

Related MCP server: Uhura

Who it is for

• People who find a phone call hard. For a deaf or hard-of-hearing owner, or an owner with a speech disability, the business is never on the owner's line. The owner reads each question on the Echo Show and answers with a tap. Ambient makes the call.

• People who are busy when businesses are open. The call holds only for the decisions that need the owner. One tap or one sentence answers it, and Ambient does the rest of the call.

• Households and businesses that do not share a language. The owner reads English on the Show while Ambient speaks Spanish to the business.

• The business on the other end. It hears that the caller is an AI before anything else. It gets a written receipt of what it agreed. "Take us off your list" stops every later call from Ambient.

How it compares

"Not described" means that the pages we fetched on 2026-10-18 do not say. The reference bar holds the sources, and adds Pixel Call Screen and Apple's Call Screening and Hold Assist.

Run it

Five-minute judge path: run pnpm demo, open /sim, tap "Call Joe's Plumbing", and watch Ambient refuse "Yes, plus parts." Then type your own answers into "Try to fool it". Then tap "Start a new demo call" and "Get 3 quotes at once" to watch three calls run together. Then run pnpm smoke:mcp, pnpm smoke:around, pnpm call:dry and pnpm scenarios.

The demo needs no keys and no accounts. The business is simulated. Every action is a real MCP tool call. Measured on 2026-10-19 (Windows 11, warm pnpm store): pnpm install took 20 seconds from a fresh clone, and pnpm demo served /sim 2.5 seconds after it started. The first install on a cold store took 122 seconds.

corepack enable
pnpm install
pnpm demo
# open http://127.0.0.1:3000/sim and tap "Call Joe's Plumbing"

pnpm smoke:mcp drives the same call through /mcp from the command line. It fails unless the call ends confirmed at the counter the owner approved. Joe counters the owner's offer, so the owner is asked a second time. At the read-back, Joe answers "Yes, plus parts." Ambient does not book. It asks for the total and reads back again:

> delegate {"task":"Call Joe's Plumbing and book a visit to fix the kitchen sink, keep it under 150","newConversation":true}
  Ambient reply: Joe's Plumbing wants one hundred eighty dollars for Tuesday the 20th at 10 am. Your limit is one hundred fifty dollars. Say yes to book it, offer a price like one fifty, or say no.
> status
  Ambient reply: Joe's Plumbing is holding at one hundred eighty dollars, over your limit of one hundred fifty dollars. Say yes to book it, offer a price like one fifty, or say no.
> delegate {"task":"offer 150"}
  Ambient reply: Okay, offering one hundred fifty dollars. To hear their answer, ask how it's going.
> status
  Ambient reply: Joe's Plumbing turned down your offer and is holding at one hundred sixty dollars, over your limit of one hundred fifty dollars. Say yes to book it, offer a price like one fifty, or say no.
> approve {"decision":"once"}
  Ambient reply: Joe's Plumbing confirmed Tuesday the 20th at 10 am for one hundred sixty dollars.
> status
  Ambient reply: Joe's Plumbing confirmed Tuesday the 20th at 10 am for one hundred sixty dollars.

The end of the same run, from the call transcript:

  Maya: Approved: one hundred sixty dollars for Tuesday the 20th at 10 am
  Ambient: Okay, Maya says one sixty is fine. Please go ahead and book Tuesday at ten.
  Joe's Plumbing: Hmm, one sixty. Alright, that works. I'll put you down for Tuesday the 20th at 10 am.
  Ambient: Okay, so just to confirm, one hundred sixty dollars total, Tuesday the 20th at 10 am. Is that right?
  Joe's Plumbing: Yes, plus parts.
  Ambient: Sorry, I need one total price. What is the total with parts?
  Joe's Plumbing: Fair enough. A sink like that needs no parts, so that's all in.
  Ambient: Let me say it back, one hundred sixty dollars total, Tuesday the 20th at 10 am. Is that right?
  Joe's Plumbing: Yes, that's right. See you then.
  Ambient: Great, that's set for Tuesday the 20th at 10 am. Thanks so much, bye.
smoke: ok, the business countered, "yes, plus parts" did not book, and the call booked at the approved counter

The full run, with the call transcript, is in docs/demo/smoke-transcript.txt.

pnpm scenarios runs four more calls through the same reducer, with no server: a vet that gives no price (the owner is asked, and the read-back says "no price is set"), a business that never gives a clean yes (unconfirmed after three read-backs), no answer, and an owner who declines the price. Output: docs/demo/scenarios.txt.

How a call works

One pure function owns the call: step(state, input, now) in src/calls/session.ts. It takes the current state and one input (the business spoke, the brain produced words, the owner decided, the line dropped) and returns the next state and a list of effects (say these words, think, hang up, tell the engine). It has no clock, no network, and no model inside it. Every honesty rule below is a table test against it.

A thin shell, src/calls/call.ts, runs the effects. Two seams plug into it:

• The line carries speech as text. simulatedLine plays Joe's Plumbing for the demo. CallRoom is the real voice line.

• The brain decides what to say. claudeBrain streams one turn from Claude with three strict tools: ask_owner, end_call, and remember. scriptedBrain replays rules for the demo.

A call is a run in the existing engine (src/conversations.ts), so the MCP tools, the Alexa skill, and the simulator all see the same state.

Honesty rules

Rules 1 to 4 and 8 are enforced in the call reducer and tested in test/call-session.test.ts. Rules 5 and 6 are enforced in the caller runtime and tested in test/caller.test.ts. Rule 7 is enforced in the errand reducer and tested in test/errand.test.ts.

Rule 2 in detail:

• Only the brain's end_call can start a booking. The reducer then reads the terms back word for word.

• The read-back counts only after it has fully played. The voice page reports playback, and speech during playback that is not agreement cancels it.

• The reply books only if it is short, contains a confirmation word ("yes", "that's right", "sounds good"), and otherwise holds only courtesy words and the exact amount and time from the read-back. Any other word or number, a negation, or a question returns to the conversation.

• Each retry is worded differently, so a late playback report cannot arm the wrong one.

• After three unanswered read-backs, or five in all, the call ends unconfirmed. A dropped line is always unconfirmed.

• A labelled corpus of 192 replies pins the reply check. A seeded test plays 4,000 generated calls and checks that every booking followed a clean yes to a read-back of that exact amount.

• In Spanish, the reply books only if it is a clean yes ("sí", "correcto", "así es", "de acuerdo") with courtesy words and nothing else. Any number, qualifier or question returns to the conversation. The read-back needs a plain Spanish day and time that names the same day, date and hour as the English booking it records. Its price is spoken in Spanish words.

• Each booking keeps its proof: the full read-back the business heard and the reply that booked it, each with its time. The 4,000-call test checks that the proof is exactly that pair. The card quotes both lines. On a live call, "Hear them say yes" plays that moment from the call recording. AssemblyAI records each voice session, and Ambient finds the read-back in the session timeline. It plays a recording only from the agent that made this call. The simulator has no recording, so there the card speaks both lines. pnpm call:dry checks the recording path against a local fake. No real recording has played yet.

Joe says "Yes, plus parts." and Ambient does not book.

Each rule maps to tests. The names below are the exact it(...) names.

test/read-back-replies.test.ts adds the 192-reply corpus for rule 2.

Three adversarial reviews of the reducer and repeated structural review rounds, all by AI reviewers, tried to break these rules and the voice line. Each round is logged in progress.md. Each defect they reproduced is now a regression test.

The house

• Echo Dot. The custom skill (src/alexa.ts, alexa/interaction-model.en-US.json) starts, steers, approves, and ends calls in one sentence each. When a call needs the owner, a Voice Monkey announcement speaks the question and Proactive Events adds a notification (src/cue.ts).

• Echo Show and Fire TV. The MCP App card leads with the stake ("$180 · Tuesday the 20th at 10 am"). On a screen it carries "Book $180", "Offer $150", and "Decline", which call the approve and delegate tools through the host. If the business agrees a day but gives no calendar date, the card says "Booked · date still open" with "Only the weekday was given · ask for the date", and Alexa adds "They gave a day but no date, so check which one."

• Calling around. One instruction calls up to three businesses at the same time. The Echo Show shows a tile per business: ringing, on the phone, its quote, then booked or passed. The booked card names the cheapest quote and lists the passed ones.

• Fire TV. When the call needs the owner, the film pauses, the picture dims, and a corner card shows the stake. It plays on once the owner answers from any device.

• Moving a booking. Say "Alexa, move the plumber to Thursday." Ambient calls the business it last booked and asks to move the visit. The price you approved carries over, and the new time still books only through a read-back and a clean yes. A request about another trade, such as "move my dentist appointment", does not move the plumber (test/caller.test.ts).

• Calling again. If a call ends without a booking, say "call them again". Ambient redials the same business with the original job. It does not redial a call that booked or one that is still going (test/alexa.test.ts).

• Memory. Each call is stored in SQLite. The next call to the same business starts with what the last one learned ("Remembered from Oct 18 — confirmed, quoted one hundred sixty dollars, Tuesday the 20th at 10 am").

Real calls

There is no phone network in this version. AssemblyAI cannot dial out, and a paid Twilio account was out of budget. Instead, the business answers on a web link, which a phone browser opens like an incoming call:

For each call, src/calls/rooms.ts creates an AssemblyAI agent. The agent's greeting is the disclosure, spoken word for word. Its LLM is our own endpoint, /call/<id>/v1/chat/completions. That endpoint passes each business turn into the reducer and streams back exactly what the reducer says. The honesty rules therefore hold on the real voice line too. The page streams audio straight to AssemblyAI with a single-use token, so the API key stays on the server.

cp .env.example .env
# in .env: ANTHROPIC_API_KEY, ASSEMBLYAI_API_KEY and an https PUBLIC_BASE_URL
pnpm call

Verification

pnpm scan    # typecheck, eslint, knip, and the full test suite
pnpm build
pnpm smoke:mcp            # a full simulated call through /mcp
pnpm smoke:around         # three simulated businesses at once through /mcp; the cheapest under the limit books
pnpm call:dry             # the same call over the AssemblyAI voice-line code, against a local fake of AssemblyAI (no keys); the first line it prints is "brain: scripted … AssemblyAI: a local fake"
python scripts/shots-sim.py   # every call state, light and dark, into docs/media/ (start pnpm demo with AMBIENT_DATA_DIR set to an empty directory first)
pnpm brain:latency        # time to first word on Claude (needs ANTHROPIC_API_KEY)

Limits, stated plainly

• The real voice line has not yet run against the live AssemblyAI API. The request AssemblyAI sends to our endpoint is undocumented, so the endpoint reads it defensively (friction log).

• pnpm brain:latency has not run yet, so the time to the first word on a live call is not measured. Our own part is measured: in three pnpm call:dry runs on 2026-10-19, with the scripted brain over loopback, our endpoint sent its first word 3 to 6 ms (p50) and at most 12 ms after AssemblyAI's request. The honesty checks therefore add almost nothing to a turn. Claude's time to the first token and AssemblyAI's speech time come on top.

• The cost of a call is estimated, not measured: about 26 to 31 cents for the demo call, of which AssemblyAI is about 23 cents and Claude about 3 to 8 cents. The submission shows the arithmetic.

• The Echo Dot flow needs the custom skill deployed to a public HTTPS host. The AWS stack is prepared (pnpm infra:synth) but not deployed.

• The simulator stands in for the Alexa+ host. It is a real MCP client over loopback.

What is real and what is simulated

Docs

Plan · AssemblyAI reference · Alexa reference · friction log · product feedback · CONTRIBUTING