TRIPOD+AI Checklist MCP
README.md
# TRIPOD+AI Checklist MCP
<!-- mcp-name: io.github.Black-Swan-Causal-Labs/tripod-ai-mcp -->
An MCP server that operationalizes the **TRIPOD+AI** reporting guideline
(Collins et al., *BMJ* 2024;385:e078378) as a deterministic, provenanced
instrument for measuring how completely a clinical prediction model study
reports what the checklist requires — whether the model was built with
regression or with machine learning.
It answers one question, per item, with evidence: *did this paper report this?*
It does **not** judge whether the study was well done. A clearly reported bad
choice is `reported`.
## What makes this different from a rubric
**Verdicts are derived, not asserted.** TRIPOD+AI ships an official expanded
checklist (the "Explanation & Elaboration Light" supplement) that enumerates
what each item actually requires. Those enumerations are encoded per leaf as
**167 required elements**. The assessor marks every element
`present` / `absent` / `not_triggered`, and the server computes the verdict:
| element grid | verdict |
|---|---|
| every triggered element present | `reported` |
| some present, some absent | `partial` |
| every triggered element absent | `not_reported` |
| nothing triggered at all | `not_applicable` |
A verdict that contradicts its own grid is **rejected** with a message naming
the contradiction. This is the difference between a rubric an assessor can
drift against and an instrument that enforces its own rule.
## Structure
27 published items → **64 scoreable leaves**. Item 2 (Abstract) delegates
entirely to the *TRIPOD+AI for Abstracts* checklist, so it is expanded into its
13 constituent leaves (`2.1`–`2.13`) — which also means an item-2 roll-up
reproduces the 52-leaf view on demand.
**Study type sets the denominator.** The published D / E / D;E tags mean the
applicable leaf set depends on what kind of study this is:
| `study_type` | applicable leaves |
|---|---|
| `development` | 60 |
| `evaluation` | 54 |
| `both` | 64 |
This is **declared, never inferred**. A train/test split, bootstrap, or k-fold
cross-validation inside a single dataset is *internal validation* and makes a
study `development`; `evaluation` requires data differing in time, centre or
population, or a previously published model. (Encoded from Kuijten et al.,
*J Clin Epidemiol* 2026;199:112438, whose definition both reference reviews
applied.) Getting it wrong silently mis-scopes every rate you compute.
Leaves excluded by study type render as **out of scope**, visibly distinct from
`not_reported` — a reader sees the whole published checklist and why each item
carries no verdict.
## Review flow
```
parse_manuscript → SectionMap with character-offset, source-tagged spans
assess_manuscript → the rubric + prompt for YOU to run (scaffold mode)
submit_scaffold_verdicts → validates, resolves evidence, returns the STAMPED artifact
```
The stamped render returned in `report` is the **artifact of record**. Do not
hand-assemble a checklist document: it will lack the provenance stamp, which is
derived from the spec version, the hash of the exact ingested text, and the
prompt hash, and is therefore not forgeable by a document that was not produced
by this renderer.
Eleven tools: `get_checklist`, `parse_manuscript`, `parse_pmcid`,
`assess_manuscript`, `submit_scaffold_verdicts`, `render_checklist`,
`render_checklist_docx`, `render_checklist_html`, `aggregate_corpus`,
`build_coding_sheet`, `validate_against_gold`.
## Install
```bash
pip install tripod-ai-mcp
```
```json
{
"mcpServers": {
"tripod-ai-checklist": {
"command": "tripod-ai-mcp"
}
}
}
```
## Deliberate design choices
**Evidence is confined to the scored section for abstract leaves.** The
abstract is a self-contained reporting unit under TRIPOD+AI, so a quote that
resolves outside it does not satisfy an abstract leaf; such quotes are flagged
in `off_section_evidence_leaves`.
**External artifacts are judged on the pointer, not the contents.** Several
leaves (18c–18f, 22, 12g) are satisfiable by pointing at a repository,
protocol, registration, model object or API. The server never fetches those.
It scores whether the pointer is specific and resolvable — a URL, DOI,
accession, or registration number. "Data available upon reasonable request" is
named in the guideline itself as a platitude to avoid, and does not satisfy the
element. Every render states this limit.
**Six leaves are binary by construction.** `2.4`, `2.8`, `2.10`, `2.12`, `18b`
and `27c` carry a single required element, so `partial` is arithmetically
unreachable for them. Their rubrics say so explicitly, and a spec-load
invariant enforces that they keep saying so.
**Item wording is ours, not the guideline's.** The render shows this
instrument's own re-worded item labels, not the published checklist wording,
because no explicit redistribution licence for the TRIPOD+AI checklist could be
verified when this was built. The render says so and points at
[tripod-statement.org](https://www.tripod-statement.org/) for the authoritative
text. If that licence is confirmed, populating `OFFICIAL_TEXT` in
`render.py` switches the item column over with no other change.
## Validation status
Validated against human reference coding on a small corpus. **Read the caveats
before quoting any rate.**
- **Reproducibility.** Two independent blind scorers agree on 0.966 of cells
(505 cells, 9 papers, spec 0.2.0).
- **Structural fidelity.** 131 of the spec's 136 required elements correspond
one-to-one with the guideline authors' own adherence tool (de Kanter et al.,
*J Clin Epidemiol* 2026), and tests enforce that. The other five carry written
justifications.
- **Against the reference review** (Zamagni et al., *Am J Obstet Gynecol MFM*
2026;8:101862, Table 3): raw agreement about 0.68, specificity 0.93,
sensitivity 0.54 for "fully reported".
- **What the disagreement is.** All the cells where both scorers agreed with each
other but not with the reviewers were adjudicated by a human. Of the 83
disagreements:
- 72% are deliberate differences of definition;
- 14% are instrument errors, and spec 0.2.1 fixes all 7 that it targets;
- 13% are scorer noise.
**Strictness caveat.** This instrument is **stricter than human reviewers**. It
requires every element of an item, in the way the official adherence tool does,
so "partial" is common where a reviewer would have given credit. Its completeness
rates will run lower than published reviews, and that is by design, not a fault.
The validation rests on 9 papers from one clinical area (fetal growth) and one
model family. Do not quote corpus rates from it as findings about the literature
until it has been run on a broader corpus.
Full numbers: `zamagni-corpus/FINDINGS.md` (kept outside this repository,
because the corpus papers are third-party copyrighted articles).
`build_coding_sheet` and `validate_against_gold` reproduce the agreement
statistics.
## Attribution
TRIPOD+AI is the work of Collins, Moons, Dhiman, Riley and colleagues. This
server is an independent implementation and is not endorsed by the TRIPOD
group. The encoded spec re-words item intent and requirements in its own words
and is **not a substitute for the published checklist**.
Sibling servers built on the same architecture: `target-mcp` (TARGET, for
target-trial emulations) and `robins-i-mcp`.
Apache-2.0.
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues