TransBench

🏆 Built for Built with Claude: Life Sciences — Anthropic × Gladstone Institutes × Cerebral Valley · Development Track 👨‍⚕️ Built by Dr. Kayomarz — internal-medicine physician & AI engineer · kayomarz.com

A translational research agent that turns a clinician's bedside observation into a grounded, testable, bench-ready experiment — shipped as an MCP connector for Claude Science.

Why it beats a general chatbot: it grades every hypothesis against real published papers (PubMed + Europe PMC, resolvable PMIDs), demotes textbook facts so they're never sold as discoveries, content-verifies the public dataset it proposes, and — when the evidence isn't there — ships nothing instead of hallucinating a plausible answer. The refusal is the feature.

You describe something you saw in a patient, in plain words. TransBench breaks it into the biological mechanisms that could explain it, generates falsifiable hypotheses, checks each one against the real published literature, throws out the textbook facts and the unsupported guesses, and — for whatever survives — hands you one runnable computational experiment with a concrete public dataset and a paste-ready prompt for Claude Science.

It works for any disease, drug, or mechanism — not one fixed specialty. The examples below span type 2 diabetes, resistant hypertension, melanoma, and rheumatoid arthritis, and the same pipeline handles whatever you paste in.

🎥 Demo (3-minute video)

▶ Watch the walkthrough: add-your-link-here — a real de-identified case → a grounded brief with live PubMed citations → the experiment run inside Claude Science → and the honest refusal when the evidence is too thin to ship.

Related MCP server: academic-figures-mcp

⚡ Try it yourself — no API key, two commands

Golden mode replays a real, committed run byte-for-byte — nothing to configure, no key, instant:

git clone https://github.com/kayomarz97/TransBench.git && cd TransBench && uv sync
TRANSBENCH_MODE=golden PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -c "
import asyncio
from transbench.engine import run_transbench
brief = asyncio.run(run_transbench(
    '49M, type 2 diabetes with persistent postprandial hyperglycemia despite metformin at '
    'maximal dose and confirmed adherence; elevated fasting glucagon; blunted GLP-1 response '
    'to mixed-meal testing.'))
print(brief.top_experiment.claude_science_prompt)
"

🔬 Built with Claude — deeply, not superficially

• Claude Code built the whole system — the 8-agent LangGraph engine, the MCP server, and the three rigor gates.

• Claude Science is the destination: TransBench ships as an MCP connector and emits a paste-ready claude_science_prompt that Claude Science turns into a reproducible figure.

• Three Claude tiers run the pipeline — Opus 4.8 (hypothesize + experiment design, the two quality levers), Sonnet (decompose + novelty), Haiku (grade / entail / assemble).

• Dataset-agnostic: it names and content-verifies public datasets (GEO, Tabula Sapiens) — and is built to target this hackathon's Gladstone-provided data (T-cell sequencing, regulatory-activity prediction, protein-interaction networks).

Contents

• For clinicians — what it does and why it matters

• How it works

• Without vs. with TransBench

• See it in action

• Real-world use cases

• For engineers — architecture & internals

• Install

• Run the MCP server & register in Claude Science

• Modes: live · snapshot · golden

• Reproducibility — no fake data

• Reusing Iatronix, read-only

• Tests

• Repo layout

• Future plans & scalability

• Scope, safety, and status

For clinicians — what it does and why it matters

The gap it closes. Every clinic day produces observations that don't fit the guideline — a patient who doesn't respond to a first-line drug, an unexpected lab, a pattern you can't explain. Turning that spark into a testable bench question normally means days of literature review and a conversation with a computational biologist. Most sparks never make that trip.

What TransBench does, in three steps:

1. You paste an observation in plain clinical shorthand — e.g. "52F, rheumatoid arthritis, inadequate response to methotrexate at max dose; persistent synovitis; anti-CCP positive."

2. It returns a brief. Candidate mechanisms, each labelled by how much real published evidence actually backs it (with clickable PubMed / Europe PMC / ClinicalTrials.gov citations), textbook facts flagged as "established" (so they're not dressed up as discoveries), and unsupported guesses demoted.

3. It hands you an experiment. If a mechanism is both a genuine open question and grounded in evidence, you get one runnable computational experiment — a named public dataset, an ordered protocol, explicit "confirms if / refutes if" criteria, and a prompt you paste straight into Claude Science to produce a figure.

Why it's trustworthy. The single most important behavior: when the evidence isn't there, TransBench says so and ships nothing, rather than inventing a plausible-sounding answer. In the four real runs shown below, it produced experiments for two domains and deliberately declined for two — because no hypothesis cleared the evidence bar. That refusal is the feature.

How it works

A clinician's sentence goes in; a grounded, testable brief comes out. Eight cooperating agents do the work, with three hard quality gates in the middle that anything unsupported cannot pass.

Without vs. with TransBench

A general-purpose chatbot will happily answer any mechanistic question — confidently, with citations that look real, a "novel" mechanism that's actually in every textbook, and a dataset accession that may not exist. TransBench is built to make each of those failure modes impossible. The numbers below are counted from the four real runs in snapshots/:

The 2 unverifiable datasets caught are real, and the gate rejects for different reasons: in the hypertension run the model proposed a GEO accession, the gate fetched the actual record and found it was a kidney-transplant chimerism study — wrong content — and rejected it; in the diabetes run the proposed pointer wasn't even a well-formed reference to a public dataset host. Both fell back to a pinned, guaranteed-resolvable atlas, recorded transparently in the brief's feasibility_notes.

See it in action

Real, live-captured output for a type 2 diabetes observation — every field below is served verbatim from snapshots/metabolic_t2d_golden_brief.json:

(The experiment targets hepatocytes; Tabula Sapiens is a whole-body atlas, so the single download named in the prompt includes the liver/hepatocyte cells — (immune compartment) is just the pinned fallback's default label.)

Reproduce it yourself in seconds — no API key needed (golden mode replays the committed brief):

TRANSBENCH_MODE=golden PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -c "
import asyncio
from transbench.engine import run_transbench
brief = asyncio.run(run_transbench(
    '49M, type 2 diabetes with persistent postprandial hyperglycemia despite metformin at '
    'maximal dose and confirmed adherence; elevated fasting glucagon; blunted GLP-1 response '
    'to mixed-meal testing.'))
print(brief.top_experiment.claude_science_prompt)
"

The same command works for resistant hypertension, melanoma, and rheumatoid arthritis — golden mode auto-selects the matching committed brief by the observation text (see Modes). Melanoma and RA return no experiment on purpose: no hypothesis cleared the grounding bar, so the tool declined rather than fabricate one.

Real-world use cases

Framed as what you'd actually do with it — grounded in the four domains already captured here, and generalizable to any PubMed-covered area.

• Bench-directing a treatment non-responder. T2D not controlled on metformin → TransBench grounds a metformin-transporter (OCT1/OCT3) pharmacokinetic-dissociation hypothesis and returns a single-cell experiment on hepatocyte transporter expression — a concrete next question for a lab, not a literature dump.

• Explaining a paradoxical case. Resistant hypertension despite triple therapy → an aldosterone-independent WNK–SPAK–ENaC compensation hypothesis, grounded in real Gitelman-syndrome literature (PMIDs 28003083, 25841442), with a distal-tubule co-expression experiment.

• Triaging what's worth studying. Melanoma progressing on checkpoint blockade and methotrexate-refractory RA → the tool retrieves the literature, finds no generated hypothesis is both novel and sufficiently grounded, and ships no experiment — telling you the easy mechanistic story isn't actually supported yet, which is itself the useful signal.

• A safe front door to computational biology. The output is a claude_science_prompt that a non-programmer clinician pastes into Claude Science to get a figure — the connector does the translational-informatics legwork, with citations and refutation criteria attached.

Honest scope. "Universal" means any clinical/biomedical observation — this is a PubMed + Europe PMC + single-cell-atlas research tool, not a general non-medical engine. Some areas legitimately have sparser literature for a freshly-generated novel hypothesis; the same gates that demote a thin hypothesis in one domain apply identically everywhere, which is why two of four domains here produced no experiment.

For engineers — architecture & internals

Architecture

TransBench is a standalone repo that reuses the mature grounding/retrieval stack of the Iatronix backend (med.kayomarz.com) as a read-only dependency — it imports DB-free leaf functions and never modifies Iatronix (enforced by a baseline-diff guard).

The reuse seam

A single module, src/transbench/reuse.py, imports only DB-free leaves — fetch_evidence_data, fetch_drug_data, rank_article_list, build_article_registry, grounding_stats/strip_ungrounded, has_minimum_evidence/ensure_evidence, validate_citations, create_llm, neutralize_query. It never imports run_search_graph, semantic_cache, or vector_search (those need pgvector/redis). Iatronix is installed editable (uv pip install -e <IATRONIX>/backend --no-deps) so from app.services… import … resolves to the live source tree with nothing written back.

Data contract

The engine returns a schema-valid TransBrief (src/transbench/schemas.py, Pydantic v2): request_echo, axes[], hypotheses[] (each with evidence[], supporting_count, novelty, grounded, confidence), top_experiment (dataset, dataset_pointer, protocol_steps[], confirm_if, refute_if, claude_science_prompt), deduplicated references[], contradictions_surfaced[], uncertainty_note, and a run_manifest (models, temperature, caps, per-hypothesis retrieval snapshot, token spend, timestamps). axes are free-form normalized snake_case strings, so any domain names its own mechanisms.

Determinism

temperature = 0 everywhere, belt-and-suspenders: the reused create_llm has no temperature parameter (it builds at settings.llm_temperature), so we set LLM_TEMPERATURE=0 in the env and .bind(temperature=0) on every client. temperature=0 does not make live PubMed or the LLM bit-identical, so the reproducible artifact is the experiment (named dataset + protocol + claude_science_prompt) — and golden mode makes the whole brief exactly reproducible.

MCP server

mcp_server/server.py is a FastMCP server (mcp==1.28.1) exposing three tools over streamable-HTTP (how Claude Science connects — as a local URL connector, localhost:8500) and stdio (for direct/embedded use). A run legitimately takes ~60–120s (longer cold), beyond an MCP client's single-call wait-for-result timeout, so the two generators are async (submit + poll):

• generate_experiment(observation, focus_drug="") — starts the run, returns a job_id in <1s; the finished payload is the full grounded TransBrief.

• search_grounded_evidence(question) — same engine run, returns a job_id; finished payload is a lighter grounded-evidence projection.

• get_experiment_result(job_id) — poll (<1s each) until status is "done" (payload in result) or "error", so no single call ever nears the client timeout however long the run takes.

Both generators call engine.run_transbench directly (no duplicated logic) and catch create_llm's fastapi.HTTPException (missing/invalid key, bad model) to return a clean structured error.

Cost

Per run ≈ 1 decompose + 1 hypothesize + 3 grade + 3 entailment + 3 novelty + 1 design + 1 assemble (+ ≤3 short neutralize calls) ≈ ~13 LLM calls across three tiers — Haiku (grade / entail / assemble / neutralize), Sonnet (decompose / novelty), and Opus (MODEL_DEEP: hypothesize + experiment-design — the two quality levers) — plus live PubMed and one GEO content-verification fetch. Opus lifts per-run cost, but grading (the many-call step) stays on Haiku so it's bounded; set MODEL_DEEP=claude-sonnet-4-6 to run without Opus. Hypotheses are capped at 3, abstracts at 8/hypothesis, fan-out concurrency at 3.

Install

Requires Python ≥ 3.11 and uv.

git clone <repo> && cd transbench
uv sync                                  # installs everything; runs on the vendored Iatronix copy
                                         # (src/vendored/) — no external med-ai-project needed

# (optional) develop against the LIVE Iatronix source instead of src/vendored/
# (Path A, REUSE_SOURCE=installed_iatronix):
uv pip install --no-deps -e /path/to/med-ai-project/backend

Copy .env.example to .env and fill in your own keys — never commit real keys (.env is gitignored):

Use it with Claude Science (fully local)

The private, local way is to use TransBench alongside Claude Science — one command gives you a grounded brief plus a paste-ready prompt:

bash mcp_server/ask.sh "33F, resistant hypertension on telmisartan + thiazide + CCB; raised CRP"

Copy the printed claude_science_prompt block into a Claude Science chat → CS loads the dataset and produces the reproducible figure. Nothing is exposed on a network.

Prefer a real connector (CS agent calls the tool)? CS's Local command connector can't work for a tool that itself calls an LLM (its sandbox returns 403 for api.anthropic.com → the 402/403 wall), and Remote requires a public https URL (safeFetch rejects http:// + localhost). So expose the local server at a private HTTPS URL via a tunnel — nginx + Cloudflare (proxied origin, secret-path, origin locked to Cloudflare IPs) or cloudflared. Standalone "set up your own tunnel" guide + the one gotcha (proxy_set_header Host 127.0.0.1:8500 — TransBench rejects other Hosts) is in CLAUDE_SCIENCE_SETUP.md.

⚕️ Healthcare note: TransBench is not offline — reasoning is Anthropic's cloud API and it queries PubMed, so submitted observation text is sent to Anthropic's API. De-identify before submitting. The server/orchestration stay local; the LLM does not.

Modes: live · snapshot · golden

TRANSBENCH_MODE (read from the process env inside run_transbench, so it applies to every caller including the MCP tools) toggles how output is sourced:

Golden mode auto-selects. With TRANSBENCH_GOLDEN_BRIEF unset, golden mode scans snapshots/*_golden_brief.json and serves the one whose request_echo matches your observation (normalized) — so the committed hypertension, diabetes, melanoma, and RA briefs all "just work", and dropping in a new domain golden needs zero code changes. Setting TRANSBENCH_GOLDEN_BRIEF explicitly pins a single file (legacy behavior). Either way, a brief is only ever served for a matching observation — a mismatch transparently falls back to the live pipeline, so golden mode can never serve the wrong brief.

Reproducibility — no fake data

• Everything shown is real. The data cards' numbers come from committed snapshots/*.json; the flow / pipeline / architecture diagrams encode the real system structure. All figures regenerate offline via python docs/generate_readme_assets.py.

• Anyone reproduces the demos, free. TRANSBENCH_MODE=golden replays the committed briefs byte-identical with no API key.

• Live results are close, not identical. Live runs re-derive results against today's PubMed, so citations shift as the literature grows — the same observation gives close-enough results for anyone, and the experiment (named dataset + protocol + prompt) is the durable, rerunnable artifact. Every run records its models, queries, PMIDs, and dataset pointer in run_manifest.

Reusing Iatronix, read-only

TransBench never edits Iatronix. A baseline-diff guard (tests/test_iatronix_untouched.py, run under PYTHONDONTWRITEBYTECODE=1) snapshots git -C <IATRONIX_PATH> status --porcelain at the start of every test session and hard-fails on any new delta (not an absolute-empty assertion — Iatronix legitimately carries unrelated untracked files). Verified clean across every phase of this build, including every live pipeline run.

Tests

206 tests across 15 modules — mostly fully offline/deterministic (fake-LLM doubles, pure functions, or free NCBI-only calls). The handful of live Anthropic tests share one flagship pipeline run via a session-scoped fixture and skip cleanly without a key:

PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m pytest -q tests/

The load-bearing guards: test_iatronix_untouched (read-only enforcement), test_grounding + test_novelty (the rigor gates), test_universal_domains (every PubMed query anchors on the observation's own disease, never "hypertension"), test_snapshot_toggle (live/golden/snapshot + golden auto-select), test_mcp_parity (the MCP tools faithfully pass the engine's brief), and test_cost (≤3 hypotheses, batched entailment). Per-phase acceptance tests cover decompose→assemble.

Repo layout

transbench/
├─ src/transbench/     # config, schemas, prompts, reuse seam, 8 agents, rigor, LangGraph engine
├─ mcp_server/         # FastMCP server (stdio + HTTP), run scripts, connector manifest
├─ snapshots/          # committed golden briefs (hypertension, T2D, melanoma, RA) + retrieval snapshot
├─ docs/               # generate_readme_assets.py + img/ (SVG cards & diagrams, generated offline)
├─ tests/              # fixtures + 206-test suite
├─ BUILD_SPEC.md       # full design spec        KICKOFF.md  # phase-by-phase build plan
├─ CLAUDE_SCIENCE_SETUP.md   PLAN.md   .env.example

Future plans & scalability

Roadmap — not shipped yet. TransBench today is a single-run connector. The items below turn it into a durable, multi-user, clinic-ready tool. None of them change the current engine's behaviour or its safety posture: each is additive, built the same way the core was (plan first, verify-gated), and stays standalone — it never touches Iatronix.

Text it like you'd text a colleague — Slack, WhatsApp, Telegram, or any messaging app. A clinician shouldn't need a terminal. In plain terms: send an observation from the messaging app you already use, and the grounded brief comes back as a reply. Technically: a thin, stateless gateway maps an inbound message to generate_experiment (async submit + poll) and formats the returned TransBrief back — the engine is unchanged, because it already speaks MCP over HTTP.

Batch processing. In plain terms: hand it a whole list — a clinic's backlog of interesting cases — and get a brief for each, overnight. Technically: a queue in front of the existing async job API, fanning out under the current concurrency caps, deduplicated by observation hash, resumable, with results written to the brief store below. It reuses the submit/poll design; no run logic is duplicated.

Scrub patient identifiers before anything is stored (de-identify on input). This is the single biggest gate between a demo and a real clinic deployment. In plain terms: before a single word is saved or sent, names, MRNs, dates, and other identifiers are stripped, so only the de-identified clinical picture is ever kept. Technically: a de-identification guard at the ingress boundary (PHI/PII detection + redaction) that runs before persistence and before any model call; only de-identified text plus a one-way hash (for deduplication) is ever stored.

Healthcare-grade compliance. In plain terms: the guardrails a hospital needs before it will trust a tool. Technically: encryption at rest and in transit, per-user authentication and access control, an immutable audit log (the run_manifest is already an audit seed), data-retention controls, and signed agreements (BAAs) with every processor (model provider, hosting) — with region-pinning and no-training guarantees on any PHI-adjacent path.

Persistence — from a one-shot tool into a compounding knowledge asset. In plain terms: save every brief so the tool remembers, spots duplicates, and can tell you later when new evidence appears. Technically: an append-only, content-hash-keyed brief store with two derived indexes — relational/JSONB for filtering and a small vector index over observations and hypotheses for semantic dedup. It gets its own lightweight store and never reuses the Iatronix database. This unlocks:

• Re-grounding watchlist — a stored hypothesis becomes a standing query; retrieval re-runs on a schedule and the messaging front-door notifies you when a previously-ungrounded question finally clears the evidence bar.

• Institutional memory & portfolio view — a lab accumulates a searchable corpus of observation → hypothesis → experiment, and a PI sees every generated experiment ranked by grounding / novelty / feasibility.

• Feedback loop — capture which experiments were actually run and whether they confirmed or refuted, to finally measure the tool's real hit-rate.

Meet teams where they already work. In plain terms: drop briefs straight into the tools labs already use. Technically: export to an ELN (Benchling / LabArchives), a "hypothesis backlog" in an issue tracker or Notion, and an optional read-only dashboard over the store.

Phased rollout: A persist + de-identification guard → B semantic search & dedup → C re-grounding watchlist → D portfolio & feedback → E integrations, messaging, and batch.

The honest caveats carry forward at every phase: TransBench generates hypotheses, not answers; grounded ≠ correct; it is not clinical; and sparse domains legitimately yield less.

Scope, safety, and status

Complete and shipped. generate_experiment returns a grounded, cited TransBrief whose top_experiment is a runnable single-cell analysis naming a content-verified, resolvable dataset with a claude_science_prompt; the MCP server serves it over streamable-HTTP (what Claude Science connects to, via a private https tunnel) and stdio (direct clients); the pipeline is domain-universal; the Iatronix baseline-diff guard shows no new delta. Claude Science actually executing a prompt is a demo-day path (the beta app, external to this repo), with the local ask.sh + manual-paste fallback above.

Research hypothesis generation only. Not clinical, diagnostic, or prescribing advice.