Skip to main content
Glama
Reviewer Zero

Reviewer Zero

by Rishab-Ghosh

Reviewer Zero

An open-source MCP server that reviews your ML paper before submission the way a strict PI would: format and anonymization, every reference, prior work for each claim, a methodology checklist and a writing check, with every quote checked against your PDF. It runs inside Claude Code or Claude Desktop, on your own Claude plan. It never gives a score, a rating or an accept/reject verdict, and it never rewrites your text.

MIT licensed. Package reviewer-zero (import name reviewer).

Install

You need uv and a free index key. Get a free key at https://tryreviewerzero.com. Questions: hello@tryreviewerzero.com. The key meters use of the hosted search index; it is not a payment method.

Claude Code

claude mcp add reviewer-zero -e REVIEWER_ZERO_INDEX_KEY=<your key> -- uvx reviewer-zero
# the review workflow (the skill), so Claude knows the order and the rules:
git clone https://github.com/rishab-ghosh/reviewer-zero
mkdir -p ~/.claude/skills && cp -r reviewer-zero/skills/reviewer-zero ~/.claude/skills/

Then ask: "Review my paper at ~/papers/draft.pdf for ICLR 2027."

Claude Desktop: add to claude_desktop_config.json (Settings → Developer → Edit Config):

{
  "mcpServers": {
    "reviewer-zero": {
      "command": "uvx",
      "args": ["reviewer-zero"],
      "env": { "REVIEWER_ZERO_INDEX_KEY": "<your key>" }
    }
  }
}

and add skills/reviewer-zero as a skill (zip the folder and upload it in Claude's skills settings), or paste skills/reviewer-zero/SKILL.md into a project's instructions.

Optional, recommended: GROBID for the best reference parsing (and required by review_paper). It runs locally; your PDF never leaves your machine for it:

docker run --rm -p 8070:8070 grobid/grobid:0.9.1-crf

Without it, check_citations reads the reference list from the PDF's text layer and resolves about 72% as many references (see "Measured" below).

Related MCP server: CiteCheck MCP

Two ways to review

The default: the skill, on your Claude plan, no API key. Claude reads your paper and writes the review following skills/reviewer-zero/SKILL.md; the tools do retrieval and the checks that need code. Every quote goes through verify_quotes before Claude may use it.

The measured pipeline: review_paper, on your own ANTHROPIC_API_KEY. Our prompts and models, the pipeline we measured on the dev set. It needs a local GROBID and the key in the server's environment (-e ANTHROPIC_API_KEY=...). It always shows an estimate and a hard spending limit first (confirm=false spends nothing) and runs only after you agree. Two novelty settings:

novelty=

Candidates per claim sent to the reranker

Dev recall@10

Demo paper, end to end

Demo paper, cost

full (default)

150

43.2% (32 / 74)

172 s

$1.61

lite

60

36.5% (27 / 74)

≈ 2 min

≈ $0.88

Recall is on the same 50 dev cases (74 prior works named by real reviewers); times and costs are one run each on our 4-page demo paper (docs/demo), so a long paper takes longer.

Timeouts and the cheap resume. Claude Code stops waiting for a tool after its MCP tool timeout; start it with a longer one for review_paper:

MCP_TOOL_TIMEOUT=900000 claude

If a call still times out, call review_paper again on the same PDF: finished steps are served from the local cache (~/.cache/reviewer-zero) and only the unfinished ones are paid for.

Tools

Tool

What it does

Needs

Sends off your machine

check_format(pdf_path, venue, stage)

desk-reject risks: names, emails, self-citations, identifying URLs, PDF metadata, page limits, required sections

nothing

nothing

check_citations(pdf_path)

resolves every reference; flags unresolved, wrong year, wrong authors, arXiv-now-published, duplicates

index key

titles, DOIs and arXiv ids of the works you cite

find_prior_work(queries, before, k)

40 candidates for one claim from 6–10 component queries, with BibTeX

index key

the queries Claude writes

paper(key), citations(key)

one paper's details; one hop of its citation graph

index key

the paper key

verify_quotes(pdf_path, quotes)

checks each quote appears verbatim in your PDF, with its page

nothing

nothing

review_paper(pdf_path, venue, confirm, novelty)

the measured pipeline (above)

index key, ANTHROPIC_API_KEY, local GROBID

your paper's text to Anthropic under your key; queries and cited titles as above

Quota. A key allows 600 units an hour and 5,000 a day. One find_prior_work call costs about 17 units (6–10 searches plus two expansion calls), so about 35 calls an hour.

Privacy

  • Reviewer Zero's tools never send your PDF or its text anywhere, except to Anthropic under your own key in review_paper. (In the skill path, Claude itself reads the paper through your Claude client, as with any file you give it.)

  • Only search queries, paper keys and the titles, DOIs and arXiv ids of works you cite go to our index; cited works' identifiers may also go to OpenAlex, Crossref and arXiv to resolve them.

  • GROBID is used only at a local address (localhost); any other GROBID_URL is ignored.

  • No telemetry. The index counts requests per key and stores no query text.

  • tests/test_mcp_privacy.py runs every tool on a test paper and fails if any request goes to another host or carries a sentence of the paper's body.

Measured

Dev-set numbers, measured on 2026-10-02 unless noted; details, caveats and suggested wording in docs/MEASURED_CLAIMS.md.

  • Prior-work recall@10, measured pipeline (review_paper): 43.2% full, 36.5% lite, on 50 dev cases with 74 reviewer-named prior works.

  • Prior-work recall, MCP path (Claude writes the queries from the tool description, then ranks the 40 candidates): 25.7% with Sonnet 5, 27.0% with Opus 5.5 (recall@10, same 50 cases; today's index). The gap to the pipeline is mostly retrieval: the pipeline reranks 150 candidates per claim, the tool returns 40. For the most thorough prior-work search, use review_paper.

  • Skill path, end to end (Opus 5.5, 6 dev papers): about 4 minutes per paper; 184 of 188 quotes Claude submitted were verified (it must fix or drop the rest); no score or verdict sentence in any review. Too few papers for a recall number.

  • find_prior_work latency: median 5.2 s for 6 queries against the hosted index (16 calls from Austin, TX, 4.7–9.3 s): about 2.4 s of searches, 2.3 s of citation-graph expansion and 0.3 s of metadata, with at most 4 index requests in flight (the per-key limit). On a local copy of the index the same call takes a median 1.6 s.

  • check_citations without GROBID: 72.3% as many references resolved as with GROBID (30 dev papers).

  • Index: about 1.09 million papers (577k arXiv papers in ML and adjacent fields since 2017, plus works they cite and venue papers not on arXiv), 28.3 million citation edges, updated nightly.

Licence

MIT (LICENSE). Runtime dependencies are permissively licensed: mcp, anthropic, pydantic, pydantic-settings, typer, pyyaml, pdfplumber (MIT); httpx, lxml, pypdf, numpy (BSD); the optional docling extra (MIT). GROBID, run separately as a container, is Apache-2.0.

Development

make setup        # uv sync, pre-commit install
make lint test    # ruff and pytest; tests never touch the network

The pipeline behind review_paper is in src/reviewer/: parse/ (GROBID), claims/, novelty/, methods/, writing/, citations/, format/, meta/ (including the leak filter that keeps scores and verdicts out of every output), review/ (orchestration, budget) and mcp/ (the server and its tools). Prompts are versioned files next to each step.

Available Tools

7 tools
check_citationsA

Check every reference of a paper PDF: resolve it (the Reviewer Zero index first, then OpenAlex, Crossref and arXiv) and flag references that cannot be found, have the wrong year or wrong authors, are cited as arXiv preprints although now published, or are listed twice. No LLM is used and no API key other than the index key is needed.

Sent off your machine: the titles, DOIs and arXiv ids of the works the paper cites (to the index and to OpenAlex, Crossref and arXiv) — never the paper's own text. The reference list is read by a local GROBID container if one is running, otherwise from the PDF's text layer; the result says which (parser). Report each flag with the reference as printed and what it resolved to; a reference that did not resolve is not proof it does not exist.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: pdf_path (required). Returns: Output of the MCP tool check_citations (no LLM; resolution and flags only).

check_formatA

Check a paper PDF against a venue's format and anonymization rules, locally and for free (no LLM, nothing sent anywhere). venue: iclr2027, neurips2026, icml2026, cvpr2027, acl or other; stage: submission, camera_ready or preprint. At submission, anonymization problems (author names or affiliations on page 1, emails, first-person self-citations, identifying URLs, PDF metadata) are violations. Page limits and required sections are checked only for venues whose rules are built (ICLR 2027, NeurIPS 2026, ICML 2026); for the others the result says so. Report every violation with its page and evidence; never rewrite the paper.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: stage (required), venue (required), pdf_path (required). Returns: Output of the MCP tool check_format.

citationsA

List the papers one paper cites (direction "cites") or the papers in the Reviewer Zero index that cite it (direction "cited_by"), with metadata and BibTeX for each. Use it to follow a candidate from find_prior_work one hop. limit is 1 to 200 (default 50).

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: key (required), limit, direction. Returns: key, papers, direction, truncated

find_prior_workA

Find prior work that may already contain a claim of an ML paper.

Before calling: decompose ONE claim into 6 to 10 component queries, one per technique, objective, data-construction step or problem framing. Phrase each the way a prior paper's abstract would describe that component: a declarative sentence of 10 to 40 words, not a question, not naming this paper or its method name. Set before to the paper's submission date so later work is excluded. Call once per claim.

Returns up to k (default 40) candidates in a cheap first-stage order: title, year, venue, authors, abstract, link, seed_count, the queries that found it and a BibTeX entry. The order is not a judgement. You must read the abstracts and label each candidate yourself: same (it already contains the claim), close, builds on, or different, with a one-line reason; show same and close first. Never declare the paper novel or not novel overall.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: queries (required), k, before. Returns: before, queries, candidates, n_considered, +1 more

paperA

Look up one paper in the Reviewer Zero index by key: a bare arXiv id (2106.09685) or a key returned by find_prior_work (openalex:W..., s2:...). Returns title, authors, year, venue, DOI, arXiv id, abstract, a link and a BibTeX entry. Fails with a clear message if the key is unknown.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: key (required). Returns: One paper as returned by the MCP lookups paper and citations.

review_paperA

Run the full Reviewer Zero review of a paper PDF (claims, prior work, citations with support checks, methodology, writing, format) with our prompts, using the user's own ANTHROPIC_API_KEY. It costs real money (usually about $1.50 to $2.00) and takes 7 to 12 minutes.

ALWAYS call it first with confirm=false: that spends nothing and returns the estimate and the hard limit. Show both to the user and call again with confirm=true only after they agree. If a call times out, call again with the same PDF: finished steps are served from the cache and only what is left is paid for.

novelty="full" (default) sends the 150 best candidates per claim to the reranker; novelty="lite" sends 60 (quicker and cheaper, lower recall; see the README for the measured recall and time of each).

Report findings evidence-first, each with its quote. Never give a score, rating or accept/reject verdict, and never rewrite the user's text. Without ANTHROPIC_API_KEY or a local GROBID container it explains what is missing; check_format, check_citations, find_prior_work, paper and citations work without a key.

Sent off your machine by this tool: the paper's text to Anthropic under your own key; search queries and cited works' titles/DOIs/arXiv ids to the index, OpenAlex, Crossref and arXiv.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: venue (required), pdf_path (required), confirm, novelty. Returns: Output of the MCP tool review_paper. Never contains a rating, score or verdict…

verify_quotesA

Check that quotes you intend to cite from a paper appear in its PDF word for word, and on which page. Use it on every quote before putting it in a review: a quote that is not verified must not be presented as the paper's text. Whitespace, line-break hyphenation, ligatures and punctuation are forgiven; added, dropped or changed words are not. Join separate fragments with "..." only when they appear in order, close together. Quotes need at least 5 words. Up to 50 quotes per call. Local only: nothing is sent anywhere.

Privacy: runs on your machine. Your PDF and its text never leave it, except to Anthropic under your own API key when you call review_paper. What is sent: search queries and paper keys to the Reviewer Zero index (it counts requests per API key and stores nothing else), and, for check_citations, the titles, DOIs and arXiv ids of the works the paper cites, to the index and to OpenAlex, Crossref and arXiv. No telemetry.

Inputs: quotes (required), pdf_path (required). Returns: quotes, n_verified, text_engine

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.2/5.0

    Scored across 7 tools

    Disambiguation4/5

    Each tool targets a distinct operation: searching prior work, single-paper lookup, citation-graph traversal, format checking, reference resolution, quote verification, and full review. The main overlap is that review_paper orchestrates the same steps the atomic tools perform individually, but the descriptions frame the atomic tools as composable primitives, so an agent can tell which to pick.

    Naming Consistency4/5

    check_format, check_citations, verify_quotes, find_prior_work, and review_paper follow a clear verb_noun pattern. The bare nouns 'paper' and 'citations' are the only deviations, and they remain readable and unambiguous.

    Tool Count5/5

    Seven tools is well-scoped for a paper-review assistant, with each tool earning its place: atomic checks, lookup, graph traversal, and one orchestrating full review. No redundant or filler tools.

    Completeness4/5

    The surface covers the core review lifecycle: prior-work search, paper lookup, citation following, format/anonymization checks, reference validation, quote verification, and a full pipeline. Minor gaps exist (e.g. no standalone claim-extraction or title/author search), but agents can work around them.

    Maintenance

    ActivitySlowing
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Connects Claude to OpenReview to facilitate conference reviewer and area chair workflows through the Model Context Protocol. It enables users to manage paper assignments, read submissions and reviews, and perform write actions like submitting reviews or comments using a preview-and-confirm system.
      13
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Enables auditing of every citation in a LaTeX or Markdown manuscript (English or Portuguese) against the actual passage in the cited PDF, verifying quotes literally, fetching sources from local folders, Zotero, arXiv and open-access APIs, and flagging bibliography problems such as retractions or missing DOIs. It also drafts and safely applies approved rewrites or new evidence-backed paragraphs in the .tex file with validation, encoding preservation and backups.
      19
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables Claude to audit AI/ML projects for production readiness through deterministic static analysis, check ML system descriptions, compare model configurations, and generate readiness reports or deployment checklists. It returns scored, evidence-based findings with explicit unknowns while never executing submitted code or accessing the network.
      Apache 2.0
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables MCP clients such as Claude Desktop, Claude Code, and Cursor to access and process arXiv papers by fetching their LaTeX source, with tools for full paper prompts, abstracts, section listings, and specific sections. This allows accurate interpretation of mathematical expressions and notation in equation-heavy papers.
      -