ClinicalTrials-MCP

An MCP server that grounds an LLM in the public ClinicalTrials.gov registry (API v2, no key required), built as a production-minded slice of the BioLit-MCP portfolio.

Beyond simple API wrappers, it adds the pieces a real clinical-AI platform needs: trial matching, LLM-powered eligibility extraction, a retrieval eval harness with regression detection, and per-tool observability.

Tools

Related MCP server: Clinical Trials MCP Server

Why these, for a clinical-AI role

This maps directly to the day-to-day of a clinical-AI ML engineer:

• Trial matching — the core "connect patients to the right trial" problem.

• LLM extraction — turning messy clinical free text into structured data.

• Eval infrastructure — measuring output quality continuously and catching regressions before they ship (evals/).

• Observability — metrics/logging/alerting for production tool calls (observability.py).

Quickstart

uv sync                      # or: pip install -e ".[dev,llm]"
uv run mcp dev server.py     # open the MCP Inspector to call tools interactively

Add to Claude Desktop (claude_desktop_config.json), then restart it:

{
  "mcpServers": {
    "clinicaltrials": {
      "command": "/abs/path/to/.venv/bin/python",
      "args": ["/abs/path/to/clinicaltrials-mcp/server.py"]
    }
  }
}

LLM extraction is optional — set ANTHROPIC_API_KEY (and optionally ANTHROPIC_MODEL) to enable it; without a key the heuristic parser is used.

Evals

The harness scores retrieval quality against a 20-case gold set and gates on regressions vs. a committed baseline.

python evals/run_evals.py                    # run + compare to baseline
python evals/run_evals.py --update-baseline  # record current scores as baseline

Metrics reported:

• hit@k — fraction of cases where a relevant study appears in the top k.

• avg precision — mean fraction of top-k results that are on-topic.

• regression gate — non-zero exit if hit@k drops more than the tolerance (default 5%) below baseline; wired into CI.

Note: the evals hit the live public API, which rate-limits aggressive clients. Run locally to populate evals/baseline.json and evals/results.json (the committed copies are placeholders — the scaffolding environment was rate-limited).

Resilience

The API layer wraps every call to the public registry in exponential backoff (retrying rate-limit 429s and transient 5xxs, failing fast on 4xx) and a short-lived TTL cache, so repeated lookups — e.g. the many api_get calls inside match_patient_to_trials — don't re-hit the network. See _get / _ttl_cache in server.py.

Tests & CI

pytest -q                              # 14 unit tests, network mocked — fast & deterministic
RUN_INTEGRATION=1 pytest -q \
    tests/test_integration.py          # live contract tests against the real API

The unit tests mock the network, which keeps them fast but blind to upstream contract drift — a renamed query field makes every live call 400 while every mocked test still passes. The opt-in integration tests (tests/test_integration.py) exercise the real request/response contract so that breakage is caught, not shipped.

GitHub Actions (.github/workflows/ci.yml) runs the unit tests on every push/PR and runs the retrieval evals as a separate, non-blocking regression job.

Layout

clinicaltrials-mcp/
├── server.py            # MCP tools + core API layer
├── observability.py     # logging + metrics (@track decorator, snapshot())
├── evals/
│   ├── gold_set.json    # 20 labeled retrieval cases
│   ├── run_evals.py     # scoring + regression detection
│   ├── baseline.json    # committed baseline (populate locally)
│   └── results.json     # last run output (populate locally)
├── tests/
│   ├── test_server.py       # unit tests (mocked network) — logic + retry/cache
│   └── test_integration.py  # opt-in live API contract tests
├── .github/workflows/ci.yml
└── pyproject.toml

Disclaimer

Research/portfolio project. Not a medical device; output is not clinical advice. Always verify eligibility against the full protocol with a qualified clinician.