phenoforge

Semantic value set assembly for clinical cohort definitions, over MCP.

Status: early prototype. The vocabulary loader, hierarchy expansion, hybrid (BM25 + dense) search, curated OHDSI Phenotype Library matching, an eval harness scoring all of it against curated ground truth, a thin MCP server exposing them all, and a LangGraph agent that decomposes a population description, checks curated first, and pauses for human confirmation before including anything generated — all work end-to-end against a real Athena download. See Roadmap.

Setup

Requires your own OHDSI Athena bulk download — vocabulary content carries its own license terms, so it can't be bundled with this repo.

1. Create a free account at athena.ohdsi.org.

2. Under Search, select at least ICD10CM and SNOMED as vocabularies, then click Download. SNOMED is used locally only — to resolve curated-set matching (find_curated_definition) — and is never shipped or exposed through any tool surface.

3. Unzip the download into data/athena/ (this directory is gitignored — nothing under data/ is ever committed or shipped). It contains OMOP CDM vocabulary tables (CONCEPT.csv, CONCEPT_RELATIONSHIP.csv, etc.) — Athena ships vocabulary content pre-shaped as OMOP, so no separate mapping step is needed.

uv sync
python scripts/load_vocab.py data/athena --output data/vocab.duckdb
phenoforge-mcp  # stdio MCP server; add to Claude Desktop's mcpServers config to try it

Optional: curated phenotype library

find_curated_definition needs a small local cache of OHDSI Phenotype Library cohort definitions. Like the Athena download, this content carries its own terms — the OHDSI PhenotypeLibrary GitHub repo has no confirmed LICENSE file (its R package DESCRIPTION claims Apache, but that's a manifest claim, not a verified grant for the cohort content itself) — so it's fetched into a gitignored local directory, never committed to this repo. Without this step, find_curated_definition still runs and reports why nothing matched.

python scripts/fetch_phenotype_library.py  # writes data/phenotype_library/

Bundles 8 hand-picked, diabetes/kidney-relevant cohorts (not the full library): Type 2/Type 1/ gestational diabetes, diabetic ketoacidosis, retinopathy, and chronic kidney disease.

Optional: dense (semantic) search

search_concepts fuses lexical and semantic matching when a dense index has been built; without one, it falls back to lexical-only search automatically.

python scripts/build_index.py  # writes data/concept_index.lance; downloads BioLORD-2023 on first run

Optional: eval harness

Scores each retrieval method (BM25, dense, hybrid, hierarchy expansion) against the curated demo cohorts as ground truth — hierarchical distance-weighted scoring, set-level coverage, and an over-inclusion penalty (partial credit for near-misses under the same hierarchy parent, not exact-match recall). Requires the phenotype library fetch step above; dense/hybrid scoring also needs the built index.

python scripts/run_eval.py                                 # bm25 + expand_descendants
python scripts/run_eval.py --index data/concept_index.lance # + dense + hybrid

Optional: the agent

The LangGraph agent decomposes a plain-English population description into seed terms, checks each against the curated phenotype library first, and pauses on the command line for you to accept or reject any generated (unverified) candidates before they're included. Requires an Anthropic API key (decomposition is a real Claude call) and the agent extra. scripts/run_agent.py loads a local .env automatically (gitignored — never commit real keys), or export the variable directly.

uv sync --extra agent
echo 'ANTHROPIC_API_KEY=...' > .env   # or: export ANTHROPIC_API_KEY=...
python scripts/run_agent.py "adults with type 2 diabetes and diabetic nephropathy"
python scripts/run_agent.py "..." --index data/concept_index.lance  # + dense retrieval for generated candidates

Interactive exploration

Both notebooks are exploration only, never pushed to production — reusable logic stays in src/phenoforge/.

• notebooks/explore.ipynb calls the engine directly (no MCP transport) against your real built data/vocab.duckdb.

• notebooks/evaluate.ipynb runs the eval harness and walks through the metrics with explanatory text, a method-comparison chart, and a sortable per-cohort results table.

uv sync --extra dev
jupyter lab notebooks/explore.ipynb

Related MCP server: medterms-mcp

What it does

Turns a plain-English patient population description into a defensible set of ICD-10-CM codes, where every code carries provenance — whether it came from a peer-reviewed phenotype definition, from hierarchy expansion, or from semantic retrieval that a human should check.

"adults with type 2 diabetes and diabetic nephropathy"
  → decomposes into seed clinical terms
  → checks OHDSI Phenotype Library for a validated definition for each
  → falls back to hybrid retrieval for terms with no curated match,
    pausing for human confirmation before including anything generated
  → returns a ConceptSet with per-code provenance and citations

Run it for real: python scripts/run_agent.py "adults with type 2 diabetes and diabetic nephropathy" (see Setup).

Why not an existing terminology server

Several good MCP terminology servers exist. They solve lookup — "what is code X", "map X to Y". This solves set assembly, which is the actual task in cohort definition. It also covers US ICD-10-CM, which the existing servers do not, and uses semantic retrieval rather than proxied keyword search, which fails when a population description and a code description share no vocabulary.

Architecture

Three layers — a retrieval engine, a thin MCP server, and a LangGraph agent. The MCP server and the agent are independent consumers of the same engine.

Roadmap

• v0.1 — vocabulary layer. Athena loader, DuckDB schema, hierarchy queries

• v0.2 — retrieval. BM25, BioLORD-2023 embeddings, LanceDB index, RRF hybrid scoring

• v0.3 — expansion + provenance. ConceptSet model, descendant expansion, and OHDSI PL curated matching (small hand-picked demo set — see Setup)

• v0.4 — MCP server. stdio transport, four tools (see below), Claude Desktop config

• v0.5 — eval harness. Distance-weighted scoring, coverage, over-inclusion penalty; curated demo cohorts as ground truth. Decomposition accuracy deferred to v0.7 (see phenoforge.eval) — nothing decomposes a population description yet

• v0.7 — LangGraph agent. Decompose → check curated first → generate only for unresolved terms → human confirmation gate on anything generated → assemble. Real interactive CLI (scripts/run_agent.py)

• v1.0 — packaging. PyPI, docs, validation and limitations section

Skipped: v0.6 (encoder benchmark — BioLORD vs SapBERT vs MedCPT). The agent was the more demonstrable deliverable, so v0.7 was built first; the encoder benchmark may return later.

Deferred: literature-derived phenotype algorithms (published tier), RxNorm and LOINC domains

Tools

find_curated_definition is the only source of curated provenance today, and only for the 8 bundled demo cohorts. Everything from expand_hierarchy and search_concepts is generated provenance — ungrounded, structural or lexical/semantic only, and meant to be confirmed by a human before use in a cohort definition.

License

Apache 2.0. Vocabulary content (ICD-10-CM, SNOMED) carries its own license terms from OHDSI/ Athena, separate from this repo's license — nothing from data/ is redistributed.

Not a clinical decision tool

This produces code sets for research and analytics. It does not make clinical determinations and has not been validated for patient care.