Skip to main content
Glama
amos689

paper-preflight

by amos689
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

paper-preflight

Check every reference of a paper against real scholarly records before you submit. No LLM guessing, no false accusations.

CI PyPI version MIT license paper-preflight MCP server on Glama

Python 3.11 to 3.14 Input: LaTeX, Word, Markdown and PDF Checked against six scholarly databases No LLM in the verdicts Read-only MCP tools

Tested on Windows, Linux and macOS English and Simplified Chinese

English · 简体中文 · Quick start · User guide · Releases · Feedback

paper-preflight checking the demo paper: errors for an undefined citation key, a DOI that belongs to another paper, a reference no source knows, a retracted paper and a duplicate entry key; warnings for a published preprint, two entries for the same work, a wrong year and a LaTeX-escaped DOI

Language models invent references, and copy-pasted BibTeX carries wrong years, wrong authors and dead DOIs. paper-preflight reads your paper (LaTeX, Word, Markdown, Typst, a PDF or a plain reference list) and asks Crossref, dblp, arXiv, DataCite, PubMed and OpenAlex about every cited work, with Semantic Scholar too if you have a free key, Open Library for books without a DOI, and GitHub, PyPI, CRAN, Hugging Face and OpenML for software, models and datasets:

  • Does it exist?

  • Does it match what you wrote?

  • Has it been retracted?

  • Has the preprint you cite been published since?

Every finding is backed by a record, or by every source that could know the work answering "no". When it cannot tell, it says "cannot determine" and why, instead of guessing. No language model takes part in a verdict.

Status: 1.0. On three weeks of real arXiv papers held out for the release: 0.79 false alarms per 100 references, against 409 real problems found (accuracy). Rule IDs, the JSON report, settings, commands and the Python API are kept through 1.x (what is stable). False positives are the bugs we most want to hear about: please open an issue.

Quick start

With uv nothing needs installing:

uvx paper-preflight check path/to/paper

path/to/paper is the project's folder, its main .tex file or a .bib file. To install it for good (Python 3.11 or later):

pip install paper-preflight

Extra

Adds

Install

pdf

Reading the reference list of a PDF

pip install "paper-preflight[pdf]"

mcp

The MCP server for coding agents

pip install "paper-preflight[mcp]"

support

The experimental evidence finder, with its local model

pip install "paper-preflight[support]"

The repository's demo paper cites eleven works, several of them wrong on purpose. The animation above is a real run of paper-preflight check examples/demo-paper against the live sources. In it, the correct NeurIPS paper is verified through dblp although Crossref only holds fake copies of it, the book without an identifier is confirmed by Open Library, and the Chinese book, whose script is not supported yet, is "cannot determine", not "not found".

$ paper-preflight check examples/demo-paper
paper-preflight 1.0.0 · main.tex · 12 entries, 12 cited keys

error   CIT001 main.tex:31
    Citation key 'nonexistent2023' is not defined in any bibliography file (1 use(s)).
error   REF001 refs.bib:43
    The doi of 'devlin2019bert' (10.1109/cvpr.2016.90) resolves to a different work in Crossref: "Deep Residual Learning for Image Recognition" (He et al., 2016).
error   REF003 refs.bib:66
    'lindqvist2024quantum' was not found in Crossref, dblp and Semantic Scholar, and every source responded. Check that the work exists and that its title is correct.
error   REF004 refs.bib:73
    'wakefield1998ileal' has been retracted (reported by Crossref, OpenAlex). Cite it only if the text discusses the retraction.
error   CIT002 refs.bib:128
    Entry key 'kingma2015adam' is already defined at line 47; BibTeX ignores this one.
warning REF015 refs.bib:31
    'he2015residual' cites a preprint that has been published in CVPR (2016), DOI 10.1109/cvpr.2016.90. Cite the published version and keep the eprint field.
warning CIT004 refs.bib:37
    Entries 'devlin2019bert' and 'he2016deep' look like the same work (same DOI).
warning REF013 refs.bib:51
    'kingma2015adam' gives the year 2016, but dblp records 2014, 2015.
warning REF017 refs.bib:112
    The doi of 'tacl2019example' contains LaTeX escapes: '10.1162/tacl\_a\_00276'. Write it as: 10.1162/tacl_a_00276
info    REF005 refs.bib:73
    'wakefield1998ileal' has a published correction (reported by Crossref).
info    REF090 refs.bib:95
    'zhou2016ml' could not be verified: non-Latin titles are not supported yet; grey literature without an identifier (book, report, software, web page).
info    CIT003 refs.bib:116
    Entry 'lecun1998gradient' is never cited.

References: 7 verified · 1 metadata mismatch · 1 identifier conflict · 1 not found · 1 cannot determine
5 error(s) · 4 warning(s) · 3 info

Related MCP server: citeguard

What it reads

You have

Command

A LaTeX project: its folder, main .tex or .bib; a project that ships only its compiled .bbl, as many arXiv sources do, is read from it (checked, never edited)

paper-preflight check path/to/paper

A Word manuscript: citations inserted by Zotero, Mendeley or EndNote, Word's own sources, or the reference list as typed

paper-preflight check paper.docx

Markdown, Quarto or R Markdown: [@key] against its bibliography

paper-preflight check paper.qmd

Typst: @key against #bibliography(...), a .bib or Hayagriva .yml

paper-preflight check paper.typ

A bibliography on its own: BibTeX, CSL-JSON (.json), RIS (.ris) or YAML (.yml)

paper-preflight check refs.ris

A reference list as plain text, in the common styles (APA, IEEE, ACM, Nature, Vancouver, Springer, Elsevier, MDPI, GOST, Chicago, MLA)

paper-preflight check references.txt, or - for stdin

Only the PDF (with the pdf extra)

paper-preflight check paper.pdf

Any arXiv paper: its source is downloaded to a temporary folder, checked and deleted

paper-preflight check arxiv:2607.06922

What each input can and cannot tell, in detail: docs/inputs.md.

What it checks

Every reference gets one verdict:

Verdict

Means

verified

A record of the work was found and agrees with the entry

metadata mismatch

The work was found, but the entry says something else about it: the findings name the field

identifier conflict

The entry's DOI or arXiv ID belongs to another work

not found

Every source that could know the work answered, and none has it

cannot determine

It could not be decided, and the report says why

The findings say what is wrong:

Rule

Finding

REF001

The DOI or arXiv ID points to a different paper

REF002

The DOI or arXiv ID does not exist

REF003

The work was not found in any source, and every source answered

REF004 · REF005

The work was retracted, or has an expression of concern or a correction

REF010–REF014

Authors, title, year or venue differ from the real record

REF015

A cited preprint has been formally published

REF016

The registry has a DOI the entry lacks (offered as a safe fix)

REF017

An identifier is written so that links break (10.1162/tacl\_a\_00276, …v1)

REF018

A cited arXiv preprint was withdrawn by its authors

REF019 · REF021

A linked repository, package or dataset does not exist; a linked web page is gone, with no archived copy

REF020

A preprint is cited with a venue no catalogue of journals and conferences has

REF090

Cannot determine, always with the reason (source unavailable, grey literature, …)

CIT001–CIT008

Undefined, duplicate, unused or near-duplicate citation keys; broken .bib syntax

TEX001–TEX004

Included files missing, unreadable or including each other; a remote bibliography left unchecked

RUN001 · RUN002 · CFG001

A run left incomplete by a source outage; a source answered in place of another; a suppression comment that silenced nothing

Every rule has a page: what it checks, when it can be wrong, and what to do. paper-preflight explain REF003 prints it in the terminal.

How accurate is it?

Four measurements, all against the live sources:

Measured on

Result

Real papers: 60 arXiv papers from three weeks held out for 1.0, 2,714 references, every flag reviewed by hand

0.79 false alarms per 100 references (one every three papers), against 409 real problems

Hallucinations that passed peer review: the 151 GPTZero found in NeurIPS 2025 and ICLR 2026 papers

142 flagged (94%), 9 cannot determine, none verified

HALLMARK, a public benchmark: 831 entries never used during development

Any issue: precision 98.5%, recall 93.5%, false-positive rate 1.9%

Next to five other tools, on 104 references labelled by hand in a published study

Precision 72.5%, against 31–51% for the others, at 87.9% recall

Precision comes first: a reference is called fabricated only on positive evidence, and an unanswered or ambiguous lookup is reported as "cannot determine", never as "not found". The evaluation harness and every run's summary are in evals/.

Real papers

Each week is 20 arXiv papers (cs, stat, q-bio, quant-ph and astro-ph) chosen mechanically. For 1.0, three weeks never used before were committed together, before the changes they measure, and each was run once on the same code. The gate was at most 1.5 false alarms per 100 references in each week and at most 1.0 on average. Above them are the more recent weeks that stayed under 1.0, each run once with the version of its time; every week measured, those over 1.0 too, is in evals/README.md.

Papers first submitted

Version

References

Flags

Real problems

False alarms

Unclear

False alarms per 100 references

2026-10-07..13

1.0 candidate, 1st try

991

81

76

4

1

0.4

2026-09-30..10-06

0.7.0

1,014

89

79

7

3

0.7

2026-09-23..29

0.6.0

786

48

42

6

0

0.8

2026-06-24..30

0.8.0

1,127

143

132

10

1

0.9

2026-05-13..19

1.0 candidate, 1st try

1,008

133

123

8

2

0.8

2026-05-06..12

1.0 candidate, 2nd try

950

133

126

7

0

0.7

2026-04-08..14

1.0 candidate, 3rd try

1,088

194

181

7

6

0.6

2026-04-01..07

1.0 candidate, 3rd try

763

86

80

6

0

0.8

2026-03-25..31

1.0.0

809

220

209

8

3

1.0

2026-03-18..24

1.0.0

1,007

110

105

5

0

0.5

2026-03-11..17

1.0.0

898

104

95

8

1

0.9

The three 1.0.0 weeks together

1.0.0

2,714

434

409

21

4

0.79

  • The 409 real problems: 220 cited preprints since published, 149 identifiers written so that links break, 30 entries with wrong authors, titles or years (NeurIPS 2025 papers dated 2026, invented co-authors, author fields BibTeX misreads), and 10 works that are not there as cited: 5 found nowhere, 3 identifiers that do not exist, 2 that belong to other works. A few papers' reference lists are largely invented (medical imaging, LLMs).

  • The 21 false alarms: registries' own errors (a cut or misspelt title, a garbled name, a dropped symbol) 5; groups credited as their makers ask and Zotero's "N other authors" placeholder 6; works no index holds or that a later version retitled 3; a conference paper compared in place of its journal version 2; APA's [software] and [dataset] labels after a title 2; three others. When a finding is wrong says what to do.

  • The gate took four tries. The first three averaged 1.08, 1.24 and 1.28 per 100 references, each with one week over 1.5; each try's false alarms were fixed before the next, on new weeks. The first release, 0.1.0, measured 4.5. Every week measured since is in evals/README.md.

Hallucinations that passed peer review

GPTZero published 151 hallucinated references it found in NeurIPS 2025 papers and ICLR 2026 submissions, each confirmed by its staff. Pasted as plain text, as the papers printed them:

References

Flagged

Cannot determine

Verified

151

142 (94%)

9

0

  • None of them is verified. The 9 left undecided are web pages and blog posts, a workshop no source indexes, real titles given with invented authors where several works share the title, and two references too garbled to search. Each is listed with its reason in evals/results/gptzero.md.

  • GPTZero's own tool found these, so they are the hallucinations a search can find; recall on every kind of hallucination is lower (see HALLMARK below).

HALLMARK

HALLMARK (v1.2.3) is a public benchmark of real and hallucinated BibTeX entries. Fabrication counts a wrong identifier, a work not found and no author in common; any issue also counts wrong authors, title, year or venue.

Split

Mode

Precision

Recall

False-positive rate

Coverage

test_public: 831 entries, never used during development

Any issue

98.5%

93.5%

1.9%

97.7%

Fabrication

99.0%

50.2%

0.6%

97.7%

dev_public: 1,119 entries, used during development

Any issue

97.7%

94.3%

2.1%

98.9%

Fabrication

98.2%

53.7%

1.0%

98.9%

test_public was run with 1.0.0, dev_public with 0.8.0; no rule was tuned on test_public.

  • Most of the false positives are disputed labels. Every flag on a dev_public entry labelled VALID was checked by hand: the 11 that remain are not correct citations (DOIs that belong to other papers, author lists naming people who did not write the paper, a shifted year, a truncated title). Without them, both modes reach 100% precision. The list, each with a reason one lookup confirms: evals/hallmark_disputed.toml.

  • What is still missed: author lists cut short at the end, or checked only against a preprint whose later versions add people (named in an info), and invented entries that borrow a title several real works share. Every hallucination type is broken down in evals/results/.

Next to other tools

Badalova & Mayr (2026) checked 104 references by hand and published what five tools flagged. On the same references, with their labels:

Tool

Precision [95% CI]

Recall

False flags per 100 correct references

CheckIfExist

47.7% [36.0%, 59.6%]

93.9%

47.9

HalluCiteChecker

47.4% [32.5%, 62.7%]

54.5%

28.2

Hallucinator

50.9% [38.3%, 63.4%]

87.9%

39.4

HalRef

31.2% [21.9%, 42.2%]

72.7%

74.6

RefChecker

47.1% [35.7%, 58.8%]

97.0%

50.7

paper-preflight

72.5% [57.2%, 83.9%]

87.9%

15.5

The sample is small, so the intervals are wide. The study labels a reference correct when the work exists, so five of paper-preflight's "false" flags point at real errors (a wrong author, a broken DOI). Two causes of false flags found in this data were fixed before the run above (the first run measured 62.8%); see evals/results/badalova-mayr.md.

Limits

  • Titles in scripts other than Latin (Chinese, Japanese, Cyrillic, …) are "cannot determine" for now, and so is a Chinese-language work cited by a translated English title, which the open indexes rarely hold.

  • Grey literature without an identifier (reports, theses, manuals, standards) is often "cannot determine": the open registries hold little of it. Books are asked of Open Library, software of GitHub, PyPI and CRAN, models and datasets of Hugging Face and OpenML.

  • Web pages are never judged true or false. A linked page that is gone, with no archived copy, is reported (REF021).

  • A work no open index holds is "cannot determine" when its venue is unknown to the catalogues too, but at a venue they know it can be reported as not found: read REF003's page before acting on that finding.

  • Not caught: an invented reference that borrows a real title several works share, an author list cut short at the end (an info), and a citation of a real paper for a claim it does not make (only the experimental support reads the claims).

  • It needs the network on a first run: about 0.4 s per reference on a cold cache, so a 50-reference paper takes about 20 seconds. Answers are cached, and --offline works from the cache. A source that is down, with nothing to answer in its place, makes the run incomplete (exit code 2), never "clean".

Fetch verified BibTeX

Instead of writing an entry from memory, ask for it by DOI, arXiv ID or title. Every field comes from the registry record, which a comment above the entry names:

paper-preflight bib fetch 1810.04805
% Verified with paper-preflight against dblp (conf/naacl/DevlinCLT19), 2026-10-03
@inproceedings{devlin2019bert,
  title         = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language Understanding},
  author        = {Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
  booktitle     = {NAACL-HLT (1)},
  year          = {2019},
  doi           = {10.18653/v1/n19-1423},
  eprint        = {1810.04805},
  archivePrefix = {arXiv},
}

  • Preprints: an arXiv preprint that has been published comes back as the published version, with its eprint kept (--prefer preprint for the preprint itself).

  • Titles: --title (with --author/--year if needed) lists the candidates instead of choosing when several works match.

  • Retractions: a retracted work comes with a warning.

  • Scripts and agents: --format json.

Fix the bibliography

bib fix turns findings into edits of your .bib files, taken from the verified records. It prints a diff and changes nothing until you add --apply:

paper-preflight bib fix path/to/paper --level unsafe
--- a/refs.bib
+++ b/refs.bib
@@ -48,7 +47,7 @@
   title     = {Adam: A Method for Stochastic Optimization},
   author    = {Kingma, Diederik P. and Ba, Jimmy},
   booktitle = {International Conference on Learning Representations (ICLR)},
-  year      = {2016},
+  year      = {2015},
 }

  • --level safe (the default) only fixes what cannot change which work is cited: identifiers written so that links break, and DOIs the registry has but the entry lacks.

  • --level unsafe also rewrites authors, title, year and venue from the record, removes identifiers that point to another work, and cites a preprint as its published version (the entry type, venue, year, volume, pages and DOI; the eprint is kept). Review the diff first.

  • Only the affected fields change; comments, formatting, line endings and encoding are kept. A reference nobody could find is never "fixed": only you can say what was meant.

Options and exit codes

Option

Effect

--format json / --format sarif

Machine-readable output (SARIF works with GitHub code scanning); -o writes it to a file

--fail-on warning

Make warnings fail the run too (the default is errors; never for neither)

--offline

Never touch the network; use only answers already in the local cache

--refresh

Ask every source again instead of using cached answers (after a correction, say)

--recheck

Search again now for references found too new to be indexed on earlier runs (otherwise once a day)

--details

List every finding; by default a suggestion that applies to many entries (published preprints, available DOIs) is one line

--hide-info

Leave out infos

--lang zh

Chinese messages (also chosen automatically from your locale)

Exit code

Meaning

0

Nothing at or above --fail-on was found

1

Blocking findings

2

No blocking findings, but a source was unavailable and nothing could answer in its place, so the paper cannot be called clean yet

3

Usage error

4

Internal error (please report it)

The JSON report follows a published JSON Schema (schema_version 1.0): fields may be added, but none is removed or changes meaning within 1.x. Every option, the cache and its lifetimes: docs/configuration.md.

Project settings

For a whole project, put the settings in paper-preflight.toml next to the paper (or in a folder above it, up to the repository's root), or in [tool.paper-preflight] of pyproject.toml:

ignore-rules = ["REF016"]                  # never report these rules
ignore-keys = ["internal2024*"]            # nor anything about these entries
severity = { REF015 = "info" }             # report published preprints as infos
disable-sources = ["s2"]                   # optional sources only
fail-on = "warning"                        # as --fail-on; the command line wins

check --config <path> names the file instead. The optional sources are s2, openlibrary, github, pypi, cran, huggingface, openml, web (links to web pages) and wayback; the registries that judge a reference cannot be turned off. A mistake in the file (an unknown rule or setting) is an error, never silently ignored. The JSON report names the file in run.notes, and the MCP server reads it too.

Silence a finding you have checked

A comment directly above an entry silences rules for that entry, with an optional reason:

% preflight: ignore[REF003] reason="internal technical report, not indexed anywhere"
@techreport{lab2024internal,
  ...
}

The verdict stays in the JSON report; only the finding is dropped. A suppression that silenced nothing is reported as CFG001 (info), so stale comments do not pile up (for reference rules, only after a complete online run, since offline answers and outages may leave them unrun).

Better results with free credentials

paper-preflight works without any account. These optional environment variables make it faster and more complete; their values are never printed or logged.

Variable

Effect

PAPER_PREFLIGHT_EMAIL

Crossref's polite pool: faster, more reliable lookups

OPENALEX_API_KEY

A larger OpenAlex budget for retraction checks

S2_API_KEY

Semantic Scholar as a rescue source for references nobody else found

GITHUB_TOKEN

GitHub's larger budget (5,000 requests an hour, not 60) for references to repositories

NCBI_API_KEY

PubMed's larger budget (10 requests a second, not 3) for PMIDs and PMCIDs

paper-preflight doctor shows which are set and whether each source answers right now.

Use it in your workflow

Claude Code: install the plugin. It bundles an MCP server and a skill that makes Claude check the references before calling a paper finished, fix only what is proven wrong, and never invent a reference.

claude plugin marketplace add amos689/paper-preflight
claude plugin install paper-preflight@paper-preflight

Codex, Gemini CLI, Copilot, Cursor and other agents: install the same skill (gh skill install amos689/paper-preflight paper-preflight works too). It runs the CLI when no MCP server is configured:

npx skills add amos689/paper-preflight

MCP clients (Codex, Cursor, VS Code, …): run paper-preflight mcp (the mcp extra). The tools are read-only and confined to your workspace; see docs/mcp.md.

GitHub Actions: uses: amos689/paper-preflight@main checks the paper on every push, with the report in the job summary and optional code-scanning alerts; see docs/github-action.md.

pre-commit: check citation keys and cached verdicts on every commit in seconds; see docs/pre-commit.md.

Python:

import paper_preflight

report = paper_preflight.check_paper("paper/")
for ref in report.references:
    print(ref.key, ref.verdict, [f.rule for f in ref.findings])

The results are plain, frozen dataclasses; see docs/python-api.md.

Experimental: find the passage behind each citation

support looks in each cited work for a passage that says what the citing sentence claims. It reads the work's text (the arXiv source, an open-access full text or PDF, or else the abstract) and a small local model (HHEM-2.1-open, 0.4 GB) scores the passages ranked best for the claim. It needs the support extra.

paper-preflight support path/to/paper --download-model --all

--download-model fetches the model's weights once; --all also lists the confirmed citations with their quotes. arxiv:<id> works as a target, as it does for check.

  • What it says: "confirmed", with the passage quoted word for word, or "could not confirm", with the reason (no text, only the abstract, or no passage close enough).

  • It never calls a citation wrong. On a gold set of 298 citations, 93% of its confirmations were right [95% CI 82%, 98%], but it confirms only about one real citation in six, and a low score pointed at a mis-citation less than half the time. The gold set's labels were made by AI models, not experts; see evals/results/support.md.

  • Or let your agent judge. The MCP tool preflight_cited_passages returns each claim with the cited work's best passages, for the agent to judge by the same rules, without the model. On 100 gold-set citations, a Claude agent judging from it confirmed 39% of the real ones, against HHEM's 9%, every confirmation at least partly supported (the labels come from the same model family).

  • What leaves your machine: only the cited works' identifiers, to fetch their text; the claims are scored locally.

How it works

  1. Source-first. It reads the LaTeX project as LaTeX sees it: comments, \iffalse blocks and \includeonly are respected, .aux files are used when they are fresh, and the first definition of a duplicated key wins, as in BibTeX.

  2. Identifier-first routing. DOIs go to their registration agency (doi.org tells which: Crossref, DataCite, …). arXiv IDs go to arXiv, with DataCite as a fallback, and PMIDs and PMCIDs to PubMed, which also marks retracted articles. Entries without identifiers are searched by title in dblp and Crossref; dblp through its SPARQL endpoint, since its search API is behind a bot challenge. Which source is asked what, and how politely: docs/sources.md.

  3. Field-by-field matching with guards. Titles (including earlier arXiv versions'), authors (tolerating transcriptions such as Reiß/Reis), year and venue are compared. A search result is used only when enough of these agree and no other work fits as well; known fake DOI copies are skipped.

  4. One verdict per reference, as in What it checks. "Not found" needs every required source to answer "no".

  5. Cached locally. Answers are kept in a SQLite file, so re-runs are fast and --offline works.

Design principles

  • Positive confirmation or abstain. Rate limits, outages and unindexed works lead to "cannot determine", never to "not found".

  • No LLM in the verdicts. Every verdict comes from records and fixed rules, so the same records give the same report.

  • Neutral wording. Findings state observations ("not found in Crossref, dblp and Semantic Scholar, and every source responded"), never accusations.

  • Local-first, no telemetry. Only the metadata of the cited works (DOIs, titles, authors) is sent to the public scholarly APIs above. Your manuscript never leaves your machine.

What it will never do

Help evade plagiarism or AI-text detection, scrape paywalled or bot-protected sites, recommend or "complete" references from memory, or name and shame authors.

Roadmap

  • Next: references in Chinese script, measured on held-out papers before anything ships.

  • Fewer references left undecided among grey literature (reports, theses, manuals) without an identifier.

What each release brought is in the changelog; day-to-day progress in docs/PROGRESS.md (in Chinese).

Contributing

Bug reports with a reproducible .bib entry are the most valuable contribution, especially false positives. See CONTRIBUTING.md; security issues: SECURITY.md.

License

MIT. See THIRD_PARTY_NOTICES.md for adapted code.

Available Tools

5 tools
preflight_bib_fixPropose fixes for a bibliographyA
Read-onlyIdempotent

Propose edits to the .bib files, taken from the verified records, as a unified diff. Nothing is written: apply the diff with your own editing tools.

level="safe" only fixes identifiers written so that links break and adds DOIs the registry has; level="unsafe" also rewrites authors, title, year and venue from the record, removes identifiers that point to another work and cites a preprint as its published version (keeping the eprint). Show unsafe diffs to the user before applying them. References nobody could find are never "fixed".

Use it after preflight_check has reported errors or warnings in the bibliography.

ParametersJSON Schema
NameRequiredDescriptionDefault
keysNoOnly these citation keys (default: every key with a fix).
pathNoA LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)..
levelNosafe: only broken identifiers and missing DOIs; unsafe: also authors, title, year and venue from the record.safe
offlineNotrue: answer only from the local cache and never touch the network (references not cached yet come back as unverified).

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the readOnlyHint/idempotentHint annotations: it states nothing is written and the caller must apply the diff, enumerates exactly what safe versus unsafe rewrites, warns to show unsafe diffs to the user first, notes unfindable references are never fixed, and explains the offline/cache behaviour. These are consequential side-effect and workflow facts an agent cannot infer from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core action and the no-write guarantee, then layers level semantics, the user-confirmation caveat, and the sequencing hint. Every sentence carries information; nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be described, and the description still covers the diff format, mutation scope, level trade-offs, and the preflight_check dependency. An agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description meaningfully elaborates the `level` enum beyond the schema's one-line summary by listing the specific edits each level performs and the removal semantics of `unsafe`.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Propose edits"), a precise resource (.bib files sourced from verified records) and the output form (a unified diff). The read-only framing plus the sequencing with preflight_check makes it distinguishable from every sibling at a glance.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use it after preflight_check has reported errors or warnings, which is a clear triggering condition, and it explains which level to reach for. It stops short of naming alternatives (e.g. preflight_bib_lookup) or stating when not to use it, so it is strong context without full routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

preflight_bib_lookupGet verified BibTeXA
Read-onlyIdempotent

Get a BibTeX entry for a DOI, an arXiv ID or a title, built from the registry record instead of written from memory. Give identifier, or title (with author and year when you know them).

status is "found" (use bibtex as is), "ambiguous" (several works match: pick from candidates with the user, or retry with author/year), "not_found" (ask the user for the source; do not invent one) or "unavailable" (a source did not answer; retry later). A published preprint comes back as its published version with the eprint kept; status_flags lists notices such as "retracted".

Use it when adding or replacing one entry; to check a whole bibliography use preflight_check.

ParametersJSON Schema
NameRequiredDescriptionDefault
yearNoThe publication year, to narrow a title search.
titleNoThe work's title, when there is no identifier.
authorNoAn author's family name, to narrow a title search.
preferNoFor a preprint that has been published: which version to return.published
offlineNotrue: answer only from the local cache and never touch the network (references not cached yet come back as unverified).
identifierNoA DOI (10.xxxx/...) or an arXiv ID (2401.01234). Give this or `title`.

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnly/openWorld/idempotent; the description goes well beyond by enumerating the four status outcomes and their required reactions, disclosing that published preprints return the published version with the eprint kept, and warning that status_flags can carry notices such as 'retracted'. That is exactly the behavioral context the annotations cannot provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and inputs before the status taxonomy, and nearly every sentence carries useful information. It is somewhat long for a single lookup tool, and part of the status material overlaps with the existing output schema, but there is little outright filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema already exists, the description need not restate return shapes, yet it still supplies the interpretive layer an agent needs: status handling, ambiguity resolution with the user, retraction notices, and offline/unverified behavior. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real semantics beyond the schema: the either/or relationship between `identifier` and `title`, and that `author`/`year` are supplementary narrowing hints for a title search. It does not explain the `prefer` or `offline` toggles beyond what the schema already says, so it stops short of 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get a BibTeX entry for a DOI, an arXiv ID or a title') and adds the key qualifier that it is built from the registry record rather than written from memory. This distinguishes it from a naive generation tool and from its sibling `preflight_check`, which handles whole bibliographies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly scopes use ('when adding or replacing one entry') and names the alternative for the other case ('to check a whole bibliography use preflight_check'). It also tells the agent what to do per status, including 'do not invent one' for not_found and retry later for unavailable — genuine when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

preflight_checkCheck a paper's referencesA
Read-onlyIdempotent

Verify every cited reference of a LaTeX project (or a .bib file) against real scholarly records, and check citation keys and the bibliography.

Returns a summary (error/warning counts, one verdict per reference, whether the run was complete) and the findings, most severe first, max_findings at a time; call again with offset=next_offset for more. complete: false means a source was unavailable, so the paper cannot be declared clean yet. offline: true answers from the local cache only. The first run of a paper may take a minute or two (progress is reported while the references are searched); later runs are served from the cache.

Use this first, on the whole paper. Use preflight_explain to understand a finding, preflight_bib_fix to get the corrections as a diff, and preflight_bib_lookup to get a single entry.

ParametersJSON Schema
NameRequiredDescriptionDefault
langNoLanguage of the messages: English or Chinese.en
pathNoA LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)..
offsetNoWhere to start in the findings: 0 first, then `next_offset`.
offlineNotrue: answer only from the local cache and never touch the network (references not cached yet come back as unverified).
include_infoNoAlso return info findings (suggestions), not only errors.
max_findingsNoHow many findings to return in this call (at least 1).

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (readOnly/idempotent/openWorld); the description adds the operational traits an agent needs -- findings ordered most-severe-first, max_findings paging, complete:false meaning a source was unavailable, offline:true serving cache-only, and first-run latency of a minute or two with progress reporting. This is behavior well beyond what the annotations declare.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then returns semantics, then paging/caching caveats, then sibling routing -- a natural read order with no filler sentences. The only slight overlap is the offline/cache explanation, which also appears in the schema, but it earns its place as a run-level caveat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter, zero-required tool with an output schema, the description covers everything an agent must know to call it correctly and to interpret a partial result: paging loop, cache/offline tradeoff, latency expectations, and the meaning of complete:false.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description additionally supplies the orchestration semantics for pagination (offset=next_offset to continue), the caching/offline interaction, and what a partial run means, which goes modestly beyond the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource -- verifying every cited reference of a LaTeX project or .bib file against real scholarly records, plus citation-key and bibliography checks. It is clearly distinguishable from siblings preflight_bib_lookup (single entry), preflight_bib_fix (diff) and preflight_explain (finding explanation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says "Use this first, on the whole paper" and then routes to each sibling with the condition that selects it (explain a finding, get corrections as a diff, get a single entry). Pagination usage is also given ("call again with offset=next_offset").

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

preflight_cited_passagesFind the passages a citation rests onA
Read-onlyIdempotent

For each sentence of the paper that cites key: the claim it makes and the passages of the cited work ranked best for that claim, for you to judge whether the work supports it. The first call downloads the work's text (its arXiv source, an open-access full text or PDF, or else its abstract) into the local cache.

Judge each claim conservatively. Say "confirmed" only when a passage states it (quote the passage word for word) or, for a citation set right after a name such as "Adam \cite{...}", when name_in_title is true. Otherwise say "could not confirm", with the reason: no text, only the abstract, or no passage says it. Never call a citation wrong or invented on this evidence: the text may say it in other words, or be only an abstract. evidence.level is "full_text", "abstract" or "none".

Use it only after preflight_check has verified that the work exists; it answers whether a real work supports a sentence, not whether the reference is correct.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesThe citation key whose citing sentences to trace.
pathNoA LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)..
offlineNotrue: answer only from the local cache and never touch the network (references not cached yet come back as unverified).
max_passagesNoPassages to return per citing sentence, best first (1 to 20).

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly/openWorld/idempotent), it discloses a real side effect — the first call downloads the work's text into the local cache — and enumerates the fallback ordering (arXiv source, OA full text, PDF, else abstract). It also explains output semantics (`evidence.level` = full_text/abstract/none) and a behavioral rule for how to treat weak evidence, which no annotation conveys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is three dense paragraphs rather than one, but each sentence earns its place (core output, cache behavior, judging rule, evidence levels, boundary with preflight_check). The core 'what you get back' statement is front-loaded, though the download/cache caveat interrupts before the judging guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not document return shapes, yet it still clarifies the judging protocol and evidence levels — exactly the ambiguous part an agent would otherwise get wrong. Combined with annotations covering the safety profile and full param coverage, nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so `key`, `path`, `offline` and `max_passages` are already fully documented in the schema, and the description adds little param-level syntax beyond that baseline. The one value-add is contextual: the note that uncached references under `offline` come back unverified, which reinforces what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb+resource: for each sentence citing `key`, return the claim and the ranked passages that could support it, so the agent can judge support. It also distinguishes itself from the sibling `preflight_check` by declaring what question it does *not* answer ('whether a real work supports a sentence, not whether the reference is correct'). An agent can route between the two without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit precondition ('Use it only after `preflight_check` has verified that the work exists') plus a clear exclusion ('not whether the reference is correct'), and names the alternative sibling by name. When-to-use and when-not-to-use are both spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

preflight_explainExplain a ruleA
Read-onlyIdempotent

Explain a paper-preflight rule (for example REF003 or CIT001): what it detects, its default severity, the message template and whether a fix can be applied safely. Use it when a finding from preflight_check is unclear; it does not look at the paper.

ParametersJSON Schema
NameRequiredDescriptionDefault
rule_idYesA rule ID as it appears in a finding, such as REF003 or CIT001.

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and openWorldHint=false, so the safety profile is covered. The description still adds real behavioral context by disclosing that the tool operates on rule metadata only, not on the document, and by previewing fix-applicability as part of the answer. The remaining gap is minor since the output schema carries the return shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, and the ordering is correct: what the tool returns comes first, then the usage trigger and boundary. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, read-only lookup tool with both annotations and an output schema, the description supplies everything an agent needs: the input form, the trigger condition, and the fact that it is metadata-only. Nothing required for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single rule_id parameter, so the schema already documents it. The description's example rule IDs (REF003, CIT001) mirror what the schema says and add no new syntax or format guidance. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Explain a paper-preflight rule') and enumerates the exact payload the agent receives: detection logic, default severity, message template, and fix safety. It also explicitly separates itself from the sibling by stating it 'does not look at the paper,' so an agent can distinguish it from preflight_check without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is stated as an explicit trigger plus an alternative: 'Use it when a finding from `preflight_check` is unclear.' The negative boundary ('it does not look at the paper') tells the agent when this tool is the wrong choice, which is exactly the routing information needed across the preflight_* family.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv1.0.0
    • Changedpreflight_bib_fix4 fields changed
      • addedInput schema / properties / keys / description
        Added value: +"Only these citation keys (default: every key with a fix)."
      • addedInput schema / properties / level / description
        Added value: +"safe: only broken identifiers and missing DOIs; unsafe: also authors, title, year and venue from the record."
      • addedInput schema / properties / offline / description
        Added value: +"true: answer only from the local cache and never touch the network (references not cached yet come back as unverified)."
      • addedInput schema / properties / path / description
        Added value: +"A LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)."
    • Changedpreflight_bib_lookup6 fields changed
      • addedInput schema / properties / author / description
        Added value: +"An author's family name, to narrow a title search."
      • addedInput schema / properties / identifier / description
        Added value: +"A DOI (10.xxxx/...) or an arXiv ID (2401.01234). Give this or `title`."
      • addedInput schema / properties / offline / description
        Added value: +"true: answer only from the local cache and never touch the network (references not cached yet come back as unverified)."
      • addedInput schema / properties / prefer / description
        Added value: +"For a preprint that has been published: which version to return."
      • addedInput schema / properties / title / description
        Added value: +"The work's title, when there is no identifier."
      • addedInput schema / properties / year / description
        Added value: +"The publication year, to narrow a title search."
    • Changedpreflight_check6 fields changed
      • addedInput schema / properties / include_info / description
        Added value: +"Also return info findings (suggestions), not only errors."
      • addedInput schema / properties / lang / description
        Added value: +"Language of the messages: English or Chinese."
      • addedInput schema / properties / max_findings / description
        Added value: +"How many findings to return in this call (at least 1)."
      • addedInput schema / properties / offline / description
        Added value: +"true: answer only from the local cache and never touch the network (references not cached yet come back as unverified)."
      • addedInput schema / properties / offset / description
        Added value: +"Where to start in the findings: 0 first, then `next_offset`."
      • addedInput schema / properties / path / description
        Added value: +"A LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)."
    • Changedpreflight_cited_passages4 fields changed
      • addedInput schema / properties / key / description
        Added value: +"The citation key whose citing sentences to trace."
      • addedInput schema / properties / max_passages / description
        Added value: +"Passages to return per citing sentence, best first (1 to 20)."
      • addedInput schema / properties / offline / description
        Added value: +"true: answer only from the local cache and never touch the network (references not cached yet come back as unverified)."
      • addedInput schema / properties / path / description
        Added value: +"A LaTeX project folder or .tex file, a Markdown, Quarto or Typst manuscript (.md, .qmd, .typ), a Word manuscript (.docx), or a reference list (.bib, .bbl, .json, .ris, .yml, .txt, .pdf), relative to the workspace root (default: the root itself)."
    • Changedpreflight_explain1 field changed
      • addedInput schema / properties / rule_id / description
        Added value: +"A rule ID as it appears in a finding, such as REF003 or CIT001."
  2. 5 tool updatesv0.1.0
    • First observedpreflight_bib_fix
    • First observedpreflight_bib_lookup
    • First observedpreflight_check
    • First observedpreflight_cited_passages
    • First observedpreflight_explain
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.7/5.0

Scored across 5 tools

Disambiguation5/5

Each tool targets a genuinely distinct action: whole-paper verification (check), rule explanation (explain), single-entry retrieval (bib_lookup), diff generation (bib_fix), and claim-support evidence (cited_passages). The descriptions explicitly cross-reference each other and clarify scope boundaries (e.g. 'to check a whole bibliography use preflight_check'). No two tools overlap in purpose.

Naming Consistency5/5

All tools use a consistent 'preflight_' prefix followed by a snake_case noun/verb pattern (preflight_check, preflight_explain, preflight_bib_lookup, preflight_bib_fix, preflight_cited_passages). The convention is predictable and readable throughout, with no style mixing.

Tool Count5/5

Five tools is well-scoped for a reference-preflight service, and each maps to a clear stage of the workflow (verify, explain, lookup, fix, judge support). Nothing feels redundant or missing at the count level.

Completeness4/5

The surface covers the full lifecycle: whole-paper check, rule explanation, single-entry lookup, safe/unsafe fixes, and citation-support evidence. Minor gaps exist (e.g. no bulk bibliography export or cache-management tool), but agents can work around them and core workflows are complete.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • citeguard

    wedo911
    A
    license
    A
    quality
    A
    maintenance
    Checks whether a citation has been retracted, corrected, or flagged with an expression of concern by querying Crossref — including retractions that Crossref backfills from the Retraction Watch database, which publishers often never record in their own metadata. Lets an AI agent verify a DOI, or every DOI in a reference list, before using it in research or writing.
    2
    1
    MIT

  • citation-check

    arhancanli
    A
    license
    A
    quality
    B
    maintenance
    Enables verifying citations in reference lists and BibTeX by detecting fabricated or mismatched references and retractions, and returning corrected BibTeX.
    3
    MIT

  • CiteCheck MCP

    lucasgris
    A
    license
    A
    quality
    C
    maintenance
    Enables auditing of every citation in a LaTeX or Markdown manuscript (English or Portuguese) against the actual passage in the cited PDF, verifying quotes literally, fetching sources from local folders, Zotero, arXiv and open-access APIs, and flagging bibliography problems such as retractions or missing DOIs. It also drafts and safely applies approved rewrites or new evidence-backed paragraphs in the .tex file with validation, encoding preservation and backups.
    19
    MIT