Skip to main content
Glama
TRIPOD+AI Checklist MCP

TRIPOD+AI Checklist MCP

TRIPOD+AI Checklist MCP

An MCP server that operationalizes the TRIPOD+AI reporting guideline (Collins et al., BMJ 2024;385:e078378) as a deterministic, provenanced instrument for measuring how completely a clinical prediction model study reports what the checklist requires — whether the model was built with regression or with machine learning.

It answers one question, per item, with evidence: did this paper report this? It does not judge whether the study was well done. A clearly reported bad choice is reported.

What makes this different from a rubric

Verdicts are derived, not asserted. TRIPOD+AI ships an official expanded checklist (the "Explanation & Elaboration Light" supplement) that enumerates what each item actually requires. Those enumerations are encoded per leaf as 167 required elements. The assessor marks every element present / absent / not_triggered, and the server computes the verdict:

element grid

verdict

every triggered element present

reported

some present, some absent

partial

every triggered element absent

not_reported

nothing triggered at all

not_applicable

A verdict that contradicts its own grid is rejected with a message naming the contradiction. This is the difference between a rubric an assessor can drift against and an instrument that enforces its own rule.

Related MCP server: clearon-source-audit

Structure

27 published items → 64 scoreable leaves. Item 2 (Abstract) delegates entirely to the TRIPOD+AI for Abstracts checklist, so it is expanded into its 13 constituent leaves (2.1–2.13) — which also means an item-2 roll-up reproduces the 52-leaf view on demand.

Study type sets the denominator. The published D / E / D;E tags mean the applicable leaf set depends on what kind of study this is:

study_type

applicable leaves

development

60

evaluation

54

both

64

This is declared, never inferred. A train/test split, bootstrap, or k-fold cross-validation inside a single dataset is internal validation and makes a study development; evaluation requires data differing in time, centre or population, or a previously published model. (Encoded from Kuijten et al., J Clin Epidemiol 2026;199:112438, whose definition both reference reviews applied.) Getting it wrong silently mis-scopes every rate you compute.

Leaves excluded by study type render as out of scope, visibly distinct from not_reported — a reader sees the whole published checklist and why each item carries no verdict.

Review flow

parse_manuscript      → SectionMap with character-offset, source-tagged spans
assess_manuscript     → the rubric + prompt for YOU to run (scaffold mode)
submit_scaffold_verdicts → validates, resolves evidence, returns the STAMPED artifact

The stamped render returned in report is the artifact of record. Do not hand-assemble a checklist document: it will lack the provenance stamp, which is derived from the spec version, the hash of the exact ingested text, and the prompt hash, and is therefore not forgeable by a document that was not produced by this renderer.

Eleven tools: get_checklist, parse_manuscript, parse_pmcid, assess_manuscript, submit_scaffold_verdicts, render_checklist, render_checklist_docx, render_checklist_html, aggregate_corpus, build_coding_sheet, validate_against_gold.

Install

pip install tripod-ai-mcp
{
  "mcpServers": {
    "tripod-ai-checklist": {
      "command": "tripod-ai-mcp"
    }
  }
}

Deliberate design choices

Evidence is confined to the scored section for abstract leaves. The abstract is a self-contained reporting unit under TRIPOD+AI, so a quote that resolves outside it does not satisfy an abstract leaf; such quotes are flagged in off_section_evidence_leaves.

External artifacts are judged on the pointer, not the contents. Several leaves (18c–18f, 22, 12g) are satisfiable by pointing at a repository, protocol, registration, model object or API. The server never fetches those. It scores whether the pointer is specific and resolvable — a URL, DOI, accession, or registration number. "Data available upon reasonable request" is named in the guideline itself as a platitude to avoid, and does not satisfy the element. Every render states this limit.

Six leaves are binary by construction. 2.4, 2.8, 2.10, 2.12, 18b and 27c carry a single required element, so partial is arithmetically unreachable for them. Their rubrics say so explicitly, and a spec-load invariant enforces that they keep saying so.

Item wording is ours, not the guideline's. The render shows this instrument's own re-worded item labels, not the published checklist wording, because no explicit redistribution licence for the TRIPOD+AI checklist could be verified when this was built. The render says so and points at tripod-statement.org for the authoritative text. If that licence is confirmed, populating OFFICIAL_TEXT in render.py switches the item column over with no other change.

Validation status

Validated against human reference coding on a small corpus. Read the caveats before quoting any rate.

  • Reproducibility. Two independent blind scorers agree on 0.966 of cells (505 cells, 9 papers, spec 0.2.0).

  • Structural fidelity. 131 of the spec's 136 required elements correspond one-to-one with the guideline authors' own adherence tool (de Kanter et al., J Clin Epidemiol 2026), and tests enforce that. The other five carry written justifications.

  • Against the reference review (Zamagni et al., Am J Obstet Gynecol MFM 2026;8:101862, Table 3): raw agreement about 0.68, specificity 0.93, sensitivity 0.54 for "fully reported".

  • What the disagreement is. All the cells where both scorers agreed with each other but not with the reviewers were adjudicated by a human. Of the 83 disagreements:

    • 72% are deliberate differences of definition;

    • 14% are instrument errors, and spec 0.2.1 fixes all 7 that it targets;

    • 13% are scorer noise.

Strictness caveat. This instrument is stricter than human reviewers. It requires every element of an item, in the way the official adherence tool does, so "partial" is common where a reviewer would have given credit. Its completeness rates will run lower than published reviews, and that is by design, not a fault. The validation rests on 9 papers from one clinical area (fetal growth) and one model family. Do not quote corpus rates from it as findings about the literature until it has been run on a broader corpus.

Full numbers: zamagni-corpus/FINDINGS.md (kept outside this repository, because the corpus papers are third-party copyrighted articles). build_coding_sheet and validate_against_gold reproduce the agreement statistics.

Attribution

TRIPOD+AI is the work of Collins, Moons, Dhiman, Riley and colleagues. This server is an independent implementation and is not endorsed by the TRIPOD group. The encoded spec re-words item intent and requirements in its own words and is not a substitute for the published checklist.

Sibling servers built on the same architecture: target-mcp (TARGET, for target-trial emulations) and robins-i-mcp.

Apache-2.0.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers