Clinical AI Governance Platform

Agents propose. A deterministic layer validates. A human approves. Every action is audited.

A production-grade reference implementation for deploying LLM agents over clinical data with deterministic safety guardrails. Built as a reusable framework for healthcare operators — deploy once, apply across a portfolio.

Live demo: https://clinical-ai-governance-platform-production.up.railway.app

End-to-end pipeline

Raw PDF / prior auth letter / treatment plan
  ↓
  parse_clinical_document  (Nemotron Parse — NVIDIA NIM)
  Multi-column OCR, table extraction, reading-order reconstruction
  ↓
  de-identification layer  (deidentify.py)
  Strip name/MRN, hash patient ID, bucket age — before any external API
  ↓
  extract_entities  (ClinicalNLP — Anthropic structured output, temp=0)
  ICD-10-CM · LOINC · NPI · RxNorm · calibrated confidence
  ↓
  search_guidelines  (RAG — BM25 + ChromaDB hybrid, RRF fusion)
  Evidence-based thresholds from 8 clinical guidelines
  ↓
  search_clinical_trials  (ClinicalTrials.gov v2 API — on flagged observations)
  Recruiting trials the patient may qualify for
  ↓
  propose_observation  (LOINC deterministic gate — 14 codes)
  Hard reject on impossible values · warning on clinical flags
  ↓
  ══ HUMAN-IN-THE-LOOP GATE ══
  approve_write / reject_write  (verified approver only, DUA-gated)
  ↓
  SQLite commit  (WAL mode, FK enforcement, field-level encryption)
  ↓
  SHA-256 audit chain  (tamper-evident JSONL, verify_chain())

Related MCP server: Smart EHR MCP Server

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                  Clinical AI Governance Platform                    │
│                                                                     │
│  [IN]  Nemotron Parse (NVIDIA NIM / self-hosted for PHI)           │
│        Raw PDF → structured markdown (prior auth, EOB, plan)       │
│                              │                                      │
│                              ▼                                      │
│        De-identification layer  (deidentify.py)                    │
│        Hash patient ID · strip name/MRN · bucket age              │
│                              │                                      │
│                              ▼                                      │
│        Agent SDK Orchestration  (src/clinical_agent/)              │
│   ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐       │
│   │  Reader       │→ │  RAG          │→ │  Proposal         │       │
│   │  Subagent     │  │  Subagent     │  │  Subagent         │       │
│   └──────────────┘  └──────────────┘  └──────────────────┘       │
│        PostToolUse hooks: audit logging + cost/latency tracking    │
│                              │                                      │
│        MCP Server  (FastMCP 3.x)  10 tools · 2 resources          │
│                              │                                      │
│        Deterministic Validation  (validator.py)                    │
│        LOINC registry · value ranges · unit enforcement            │
│                              │                                      │
│        Auth + DUA layer  (auth.py)                                 │
│        Principal · Approver · Data Use Agreement verification      │
│                              │                                      │
│   ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐       │
│   │ SQLite Store │  │ ChromaDB RAG │  │ Audit Chain      │       │
│   │ WAL · FK     │  │ BM25+Semantic│  │ SHA-256 JSONL    │       │
│   │ Fernet enc.  │  │ RRF fusion   │  │ verify_chain()   │       │
│   └──────────────┘  └──────────────┘  └──────────────────┘       │
│                                                                     │
│        ClinicalTrials.gov v2 · Eval harness (25 cases, LLM-judge) │
└─────────────────────────────────────────────────────────────────────┘

Build status

Performance (eval harness, smoke suite)

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
python3 scripts/seed_db.py
pytest -q

Connect to Claude Code (local stdio)

claude mcp add clinical-governance -- \
  /path/to/.venv/bin/python -m fhir_mcp.server

Connect to claude.ai (remote SSE)

Settings → Connectors → Add → https://clinical-ai-governance-platform-production.up.railway.app/sse

Environment variables

Generate an encryption key

python3 -c "from fhir_mcp.store import generate_encryption_key; print(generate_encryption_key())"

Store the output in your secrets manager as FHIR_MCP_ENCRYPTION_KEY.

Verify audit chain

python3 scripts/audit_verify.py data/audit.jsonl

Run evals

python3 scripts/run_evals.py --suite smoke
python3 scripts/run_evals.py --suite full --judge

Repository structure

src/
  fhir_mcp/
    server.py          FastMCP 10 tools + resources + prompts
    store.py           SQLite store + field-level encryption (only PHI touchpoint)
    models.py          Pydantic v2 FHIR models
    audit.py           SHA-256 hash-chain audit
    auth.py            Principal + approver + DUA verification
    validator.py       LOINC deterministic gate
    deidentify.py      De-identification layer (hash ID, strip PHI, age bucket)
    rag.py             BM25 + ChromaDB hybrid RAG
    nlp.py             Clinical NLP entity extraction
    confidence.py      Calibrated confidence scoring
    trials.py          ClinicalTrials.gov v2 API client
    parse.py           Nemotron Parse (NVIDIA NIM) client
    http_server.py     FastAPI SSE transport
  clinical_agent/
    orchestrator.py    ClinicalOrchestrator (3-subagent workflow)
    subagents.py       Reader / RAG / Proposal subagent configs
    hooks.py           PostToolUse audit + cost hook
evals/
  golden_dataset.json  25 test cases
  runner.py            Code-based + LLM-as-judge grading
  judge_prompt.py      LLM-as-judge prompt template
  mimic_cdm_eval.py    MIMIC-CDM 4-axis governance agent eval
data/
  synthetic_patients.json   Seed data
  loinc_rules.json          14 LOINC validation rules
  clinical_guidelines.json  8 evidence-based guidelines
  clinical_notes.json       4 synthetic notes
scripts/
  seed_db.py         JSON → SQLite
  audit_verify.py    Chain integrity verifier
  run_agent.py       Agent SDK CLI
  run_evals.py       Eval harness CLI
docs/
  architecture.md    Full system design + diagrams
  adr/               4 Architecture Decision Records
  scale.md           Portfolio deployment playbook
  ci.md              GitHub Actions setup

Day-by-day build log

Datasets & Evaluation Architecture

Four datasets ground the system across two sub-projects. Each is academically sourced, operates on de-identified or synthetic data, and has a dedicated evaluation methodology drawn from peer-reviewed literature.

Dataset 1 — MedQuAD

Academic source

Ben Abacha, A., & Demner-Fushman, D. (2019). A question-entailment approach to question answering. BMC Bioinformatics, 20(1), 511. https://doi.org/10.1186/s12859-019-3119-4

47,457 question–answer pairs sourced from 12 NIH websites (MedlinePlus, CancerGov, NIDDK, NINDS, GARD, and others). Covers 37 question types across common and rare diseases. License: CC BY 4.0. No PHI — all content is public NIH patient education material.

Location: evidence_pipeline/datasets/medquad.py

LLM architecture — entity linking via deterministic crosswalk Each QA pair carries a focus (condition name) and optional UMLS CUI gold label. The pipeline maps focus → CUI via ontology/cui_mapper.py — fully deterministic, no LLM in the mapping step. The LLM role is upstream: clinical question generation and metatag refinement.

Test suite — evidence_pipeline/tests/test_datasets.py Dataset structure, field validation, is_answered / has_gold_cui / is_rare_disease properties, CSV and XML format compatibility.

LLM reasoning framework — BioEL entity linking

Sung, M., Jeon, H., Lee, J., & Kang, J. (2020). Biomedical Entity Representations with Synonym Marginalization. arXiv:2005.00239. https://arxiv.org/abs/2005.00239

Implemented in evidence_pipeline/evals/entity_linking.py and runner.py. Top-k accuracy and Mean Reciprocal Rank (MRR) over the full corpus.

Dataset 2 — MIMIC-IV Discharge Summaries

Academic source

Johnson, A.E.W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T.J., Hao, S., Moody, B., Gow, B., Lehman, L.H., Celi, L.A., & Mark, R.G. (2023). MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10, 1. https://doi.org/10.1038/s41597-022-01899-x

Goldberger, A.L., Amaral, L.A.N., Glass, L., Hausdorff, J.M., Ivanov, P.Ch., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.K., & Stanley, H.E. (2000). PhysioBank, PhysioToolkit, and PhysioNet. Circulation, 101(23), e215–e220. https://doi.org/10.1161/01.CIR.101.23.e215

De-identified ICU discharge summaries from Beth Israel Deaconess Medical Center. Demo subset (100 patients): physionet.org/content/mimic-iv-demo/ — free PhysioNet account, no CITI training. Full dataset requires CITI training + signed DUA. PHI note: loader logs note_id only, never raw text.

Location: evidence_pipeline/datasets/mimic.py, evidence_pipeline/extraction/loinc_extractor.py, evidence_pipeline/pipeline/end_to_end.py

LLM architecture — deterministic extraction + governance gate 24 regex patterns extract LOINC-coded observations from discharge text (zero LLM in extraction). The HumanGate class enforces the core governance invariant: every proposed observation is queued with a full audit entry (who/what/when/why) and committed = 0 in automated mode. Human .approve() is required to commit — wiring to src/fhir_mcp/store.py in production.

Test suite — evidence_pipeline/tests/test_mimic.py, test_loinc_extractor.py, test_end_to_end.py 5 dataset tests, 14 LOINC extraction tests, 7 end-to-end tests including core governance invariant (committed == 0).

LLM reasoning framework — FACTS Grounding

Jacovi, A., Caciularu, A., Goldman, O., & Goldberg, Y. (2025). FACTS Grounding: A New Benchmark for Evaluating the Factuality of Large Language Models. arXiv:2501.03200. https://arxiv.org/abs/2501.03200

Implemented in evidence_pipeline/evals/grounding.py. Every ICD-10, RxNorm, LOINC, CUI, and NCT-ID in pipeline output is checked against its canonical source. Grounding score = attributable_claims / total_claims. Score of 1.0 = zero unattributed claims.

Outcome metric (measured, 10 synthetic notes):

Extracted 62 LOINC-coded observations from 10 synthetic discharge notes, 100% validated, 0% rejected by deterministic gate, 0 committed without human approval.

python evidence_pipeline/demo_mimic.py                                # synthetic
python evidence_pipeline/demo_mimic.py --notes-dir /path/to/mimic    # real MIMIC-IV Demo

Dataset 3 — MIMIC-CDM (Clinical Decision Making)

Academic source

Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., & Rueckert, D. (2024). Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine. https://doi.org/10.1038/s41591-024-03097-1

Hager, P., Jungmann, F., & Rueckert, D. (2024). MIMIC-IV-Ext Clinical Decision Making (version 1.0). PhysioNet. https://doi.org/10.13026/2pfq-5b68

Derived from MIMIC-IV. Evaluates LLMs on 4-axis clinical decision making given a patient presentation. Available at physionet.org/content/mimic-iv-ext-cdm/. Leaderboard: huggingface.co/spaces/MIMIC-CDM/leaderboard.

Location: evidence_pipeline/datasets/mimic_cdm.py, evidence_pipeline/evals/clinical_decision.py, evals/mimic_cdm_eval.py

LLM architecture — dual-layer CDM eval Two separate eval targets share the same CDMCase schema and CDMScore rubric:

• evidence_pipeline/evals/clinical_decision.py — grades the evidence layer: does the ontology pipeline support correct decisions?

• evals/mimic_cdm_eval.py — grades the governance agent (src/clinical_agent/orchestrator.py): does the LLM itself make correct decisions? CI uses a deterministic crosswalk-backed mock; production wires to live ClinicalOrchestrator.

Test suite — evidence_pipeline/tests/test_mimic_cdm.py 4 dataset structure tests, 4 F1 scoring unit tests, 3 CDM eval layer tests (composite ≥ 0.75 CI gate).

LLM reasoning framework — AMIE multi-axis auto-rater

Tu, T., Palepu, A., Schaekermann, M., Saab, K., Freyberg, J., Tanno, R., Wang, A., Li, B., Amin, M., Tomasev, N., Ghassemi, M., Azizi, S., Kannan, A., Chou, K., Hassidim, A., Matias, Y., Xu, Y., Singhal, K., Gottweis, J., & Natarajan, V. (2024). Towards conversational diagnostic AI. arXiv:2401.05654. https://arxiv.org/abs/2401.05654

Token-level F1 per axis against gold ICD-10 / RxNorm / LOINC / CPT labels. Composite = mean across 4 axes. CI gate: composite ≥ 0.75.

Dataset 4 — Governance Agent Eval Harness (25 golden cases)

Source: Internal synthetic dataset, no PHI. Designed against the LOINC validation rules in data/loinc_rules.json and 8 clinical guidelines in data/clinical_guidelines.json.

Location: evals/golden_dataset.json, evals/runner.py, evals/judge_prompt.py

LLM architecture — code-based + LLM-as-judge 25 cases covering accept / reject / borderline observations across 14 LOINC codes. Deterministic code-based grading (exact accept/reject match) plus LLM-as-judge for reasoning quality. Calibrated confidence scoring uses the Brier score:

Brier, G.W. (1950). Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2

Test suite — evals/runner.py Code-based accuracy + false-negative rate, LLM-as-judge reasoning quality, calibrated Brier score.

LLM reasoning framework — LLM-as-judge

Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., & Stoica, I. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. https://arxiv.org/abs/2306.05685

Ontology foundation — UMLS CUI crosswalk

All four datasets share a common ontological foundation: the UMLS Concept Unique Identifier (CUI) as the canonical hub linking ICD-10-CM, RxNorm, LOINC, SNOMED CT, and CPT-4.

Bodenreider, O. (2004). The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Research, 32(suppl_1), D267–D270. https://doi.org/10.1093/nar/gkh061

Implementation: evidence_pipeline/ontology/cui_mapper.py — 13 conditions, deterministic lookup, zero hallucination. Grounding validated by evidence_pipeline/evals/grounding.py (FACTS Grounding, Jacovi et al. 2025).

Clinical Evidence Intelligence Pipeline

A sub-project built on top of this governance platform — zero changes to any src/ file.

The governance platform answers how to safely deploy clinical agents. This companion demonstrates what those agents generate: structured, evidence-backed clinical content at scale — directly aligned with real-world evidence (RWE) generation workflows.

Clinical question (e.g. "paroxysmal nocturnal hemoglobinuria")
    ↓
evidence_pipeline/datasets/medquad.py     47,457 NIH QA pairs (CC BY 4.0)
                                          question types · UMLS CUI labels
                                          common conditions + GARD rare diseases
    ↓
evidence_pipeline/ontology/cui_mapper.py  UMLS CUI → ICD-10-CM / RxNorm / LOINC
                                          SNOMED CT / CPT-4 crosswalk
                                          deterministic lookup — no hallucination
    ↓
Live APIs                                 ClinicalTrials.gov v2 (recruiting trials)
                                          CMS Medicare Coverage Database (NCDs + LCDs)
    ↓
evidence_pipeline/demo.py                 Structured, metatagged JSON output
                                          optimised for search and retrieval indexing
    ↓
evidence_pipeline/demo_mimic.py           End-to-end outcome metric
                                          62 LOINC observations · 100% validated
                                          0 committed without human approval

Key design principle: the cui_mapper.py crosswalk is the deterministic validation gate for ontology codes — the same agent-proposes / deterministic-validates pattern as validator.py in the main platform.

Quick demo (no API key required):

# Condition evidence brief
python evidence_pipeline/demo.py "paroxysmal nocturnal hemoglobinuria"
# → ICD-10 D59.5  CUI C0028344  RxNorm 727910 (eculizumab)
# → 5 recruiting trials  CMS coverage queried  metatagged JSON output

# End-to-end MIMIC-IV pipeline (synthetic notes, zero PHI)
python evidence_pipeline/demo_mimic.py
# → Extracted 62 LOINC observations from 10 notes
# → 100% validated, 0% rejected, 0 committed without human approval