Billing Doctor 🩺

An AI support agent that diagnoses Metronome usage-billing integration problems, and bills its own users through Metronome.

"Our usage dropped to zero after a deploy." → Billing Doctor pulls the events, compares them to the billable metrics, and replies: "Your emitter sends region: "us" but the Tokens metric only accepts "US". Values are case-sensitive, so normalize the value before sending."

Why I built this

A lot of support engineering is working out why something that "returned 200" didn't do what the customer expected: SSO configs, API auth, integrations. That investigation follows a repeatable playbook, which makes it a good fit for an AI agent.

Usage-based billing has the same shape of problem, with money attached. Billing Doctor applies that support playbook to Metronome: gather evidence, run deterministic checks, let an agent reason over the results, and hand a human a root cause plus a ready-to-send reply. I'm also using it to put the Claude Certified Architect – Foundations material into practice: agent loops, tool design, hooks, MCP and evals.

Related MCP server: Token Analyzer MCP

Why it exists

Usage-billing bugs don't show up as errors. Ingest returns 200, and the usage just never lands on the invoice. Common causes are an unregistered ingest alias, a pluralized event_type, a case-sensitive property filter, a renamed aggregation key, reused transaction_ids, or timestamps in seconds instead of milliseconds. Price disputes have their own versions: a negotiated override entered with the wrong start date, or never entered at all. Billing Doctor finds these in seconds instead of an hour of manual event spelunking.

Quick start

npm install
npm run doctor -- scenarios                          # list sandbox scenarios with planted bugs
npm run doctor -- triage --scenario region-case      # offline, deterministic, no keys needed
npm run doctor -- eval                               # score all scenarios (10/10)
npm test                                             # 27 tests, including the agent loop with a scripted model

# Claude agent mode (needs ANTHROPIC_API_KEY)
npm run doctor -- agent --scenario reused-txn-id
npm run doctor -- eval --agent

# Live Metronome account (use a SANDBOX key)
METRONOME_API_KEY=... npm run doctor -- triage --live --customer <id> --txn t1,t2 --subject "..."

Full loop: a real app sending usage → Billing Doctor

demo-app/ is a small Python product ("Acme AI") that sends usage with the official Metronome SDK. It has switchable integration bugs. Run it against the bundled fake Metronome server (npm run fake-metronome) or a real sandbox, then point Billing Doctor at the run:

npm run fake-metronome                                   # terminal 1
cd demo-app && python app.py send --calls 20 --bug wrong-alias && cd ..
METRONOME_BASE_URL=http://localhost:4010 npm run doctor -- triage --live --txn-file demo-app/last_run.json

See demo-app/README.md.

Use it from Claude Code on a subscription (no API key)

.mcp.json registers the MCP server. Open this folder in Claude Code and ask: "Use billing-doctor to figure out why transaction rc1 isn't billing." Switch scenarios with BD_SCENARIO in .mcp.json. Writes stay blocked unless BD_ALLOW_WRITES=1.

Architecture

ticket ──► agent loop (src/agent/loop.ts) ──► Claude
              │  ▲                              │ tool_use
              │  └──────── tool_result ◄────────┘
              ▼
         hooks (pre/post)  ── blocks writes without human approval, records trace
              ▼
         tools (src/agent/tools.ts) ── focused, JSON in/out, errors as data
              ▼
         checks (src/checks/*)  ── deterministic diagnostics (also used offline)
                                   findings.ts: events & invoices · pricing.ts: rebuilds price from contract + rate card
              ▼
         MetronomeClient ── HttpMetronomeClient (live / fake server) | MockMetronomeClient (scenarios)

demo-app (Python, metronome-sdk) ──usage──► fake-server.ts (localhost:4010) or real sandbox
              
         submit_diagnosis ── schema-validated structured output ── stop
              ▼
         metering (src/metering/meter.ts) ── 1 usage event per investigation → OUR Metronome

See docs/M1-NOTES.md for how the loop decides when to stop, docs/PLAN.md for milestones and docs/BUSINESS.md for how this gets sold.

Not affiliated with Metronome.