Adaptyv Foundry SDK + MCP Server + ExperimentWatcher Agent

A typed Python SDK for Adaptyv Bio's Foundry lab API, a TypeScript MCP server so Claude can drive the lab in plain English, and a governed agent that drafts customer-update emails from finished results — with a human sign-off gate and a tamper-evident audit trail so nothing reaches a customer unreviewed.

Built as a take-home for an AI Engineer role at Adaptyv Bio. No live API key was available, so everything below runs against a fixture-backed mock mode that is structurally identical in shape to the real API (validated against the vendored OpenAPI spec, not just internal models).

For the "why" behind every component and design decision — written for a non-engineer building toward a technical understanding — see LEARNING_GUIDE.md. For the system diagram and end-to-end data flows, see docs/ARCHITECTURE.md. For the full build history, phase-by-phase, including every design change and why, see ROADMAP.md.

What this actually does

Adaptyv's customers submit protein sequences and get back binding-affinity and stability measurements. Today, driving that pipeline and writing the results email is a manual, engineer-mediated chore. This project:

1. Wraps the Foundry API in a clean, typed Python SDK (adaptyv/) — one client, mock=True or mock=False, identical shapes either way.

2. Exposes it to Claude Desktop/Code via an MCP server (mcp/), so anyone can create experiments, check status, and pull results by asking in plain English.

3. Runs an ExperimentWatcher agent that polls for completed results, deterministically flags anomalies (never an LLM's judgment call — a fixed policy), and drafts a plain-English customer update — which a human must review and approve before it's ever "sent." Every draft, review, and anomaly acknowledgement is recorded in a hash-chained, append-only audit log.

4. Backs all of it with an offline eval suite (evals/) that gates on deterministic guards — not vibes — so a prompt or code change that reintroduces a defect fails loudly before it ships.

Quickstart

Requires Python 3.11+ and Node.js 18+.

# Python SDK + CLI
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"

# Try it — everything below runs with no API key, against fixtures
adaptyv experiments list
adaptyv results get aaaaaaaa-0000-0000-0000-000000000001

# Run the full test suite
make test        # 170 tests

# Run the offline eval suite (deterministic guards, zero LLM calls, zero cost)
make eval         # 3/3 golden cases

# MCP server (TypeScript)
cd mcp && npm install && npm test    # 16 tests
npm run build

Point Claude Desktop's MCP config at mcp/dist/index.js to drive the lab conversationally (see docs/ARCHITECTURE.md for the exact tool list and what each does).

Watching for new results and drafting updates

# One polling cycle, mock data, stub drafter (no Anthropic key needed)
adaptyv watch --once

# Review what it drafted
adaptyv review list
adaptyv review show <draft_id>

# A human approves (or rejects) — the agent can never do this itself
adaptyv review approve <draft_id> --by alice@adaptyvbio.com

# Verify the audit trail hasn't been tampered with
adaptyv audit verify

If a result trips a critical anomaly (e.g. all sequences failed, or the positive control is out of policy range), review approve is hard-blocked until a human runs review ack — there's no code path that lets a critical anomaly reach a customer silently.

To use real Claude-drafted emails instead of the deterministic stub, set ANTHROPIC_API_KEY and pass mock_llm=false where the bridge/MCP tool expose it (see docs/ARCHITECTURE.md) — this is the only part of the system that costs money or calls a live model; everything else, including the entire eval suite, is free and offline by design.

Project layout

adaptyv/              Python SDK — the single source of truth
├── models.py          typed request/response models, schema-faithful to the real OpenAPI spec
├── transport.py       MockTransport (fixtures) / live_transport.py (real httpx client)
├── client.py          AdaptyvClient(mock=True|False) — same shape either way
├── resources/         experiments, sequences, targets, results
├── agents/            AnomalyDetector, EmailDrafter, Watcher (the governed agent)
├── governance/        ApprovalStore (HITL state machine), AuditLog (hash-chained)
├── bridge.py           the MCP↔SDK subprocess JSON bridge (`python -m adaptyv --json`)
└── cli.py              the `adaptyv` command-line tool

mcp/                  TypeScript MCP server — 8 task-shaped tools over stdio
evals/                Offline eval suite: golden set, deterministic guards, eval→improve
                      loop, human-feedback flywheel
tests/                170 tests covering every module above
docs/                 Architecture, design specs, phase-by-phase implementation plans

Honest scope and limitations

This is a take-home, not a production system, and it's been reviewed hard (three rounds of external code review, each fixing real defects the previous round missed or introduced — the full history is in ROADMAP.md's Change Log, including what was found and how it was fixed). What's still explicitly out of scope, by design:

• No live LLM-judge eval. The eval suite gates on deterministic guards only (fact-grounding, anomaly-rule matching, HITL hard-block enforcement) — a rubric-scored LLM judge was scoped as a stretch goal and deliberately not built, since it requires live, costed Anthropic API calls that shouldn't happen by default in an offline test suite.

• Concurrency is single-connection. The governance store (ApprovalStore, AuditLog) is safe for one process/connection at a time; it does not yet handle two reviewers or two watcher processes racing on the same sqlite connection from different processes.

• The subprocess bridge trusts its caller. python -m adaptyv --json is meant to be spawned by the bundled MCP server, not exposed to untrusted input directly; the MCP tool that drafts customer updates lets the calling model choose the governance database path and whether to call a live LLM, which is a reasonable tool-surface choice for a local, single-user desktop tool but would need tightening before any multi-tenant use.

• MockTransport doesn't implement pagination/filtering. It always returns the full fixture set; only LiveTransport genuinely paginates against a real server.

• A handful of smaller, explicitly logged items (live-transport network-error mapping, a few incomplete response models) are tracked in ROADMAP.md rather than silently left unmentioned.

Testing philosophy

Every module was built test-first. make test runs all 170 Python tests; make eval runs the offline agent-output eval suite; cd mcp && npm test runs the 16 TypeScript tests. All three are fully offline — no network calls, no API key, no cost — which is also true of every CI-relevant check in this repo.