Skip to main content
Glama
Sign In
Paper Fact Check

Paper Fact Check

by Biajin-PKU
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

check_paper

Fact-check research papers by recomputing statistics, comparing text with tables, verifying references and figures, and flagging declaration or AI-writing issues.

Instructions

Fact-check a research paper (PDF, .docx, LaTeX file/folder/zip, Markdown or text) against its own evidence: recomputed statistics and percentages, text against tables, reference lookups (existence, retraction), figures, declarations and AI-writing residue. Writes findings.json, manuscript.txt and report.html to out_dir and returns a summary.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "path": "<path>"
    },
    "name": "check_paper"
  }
}

Input Schema

NameRequiredDescriptionDefault
codeNooptional folder with the released code or data
langNo
pathYesabsolute path to the manuscript
offlineNoskip reference lookups
out_dirNoreport folder (default: beside the manuscript)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.1

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: it discloses the three artifacts written (findings.json, manuscript.txt, report.html) and that a summary is returned, so the agent knows the call has persistent side effects on disk. Remaining gaps are the lack of any note on overwrite behavior, runtime cost, or whether reference lookups constitute external network calls (only implied by 'offline').

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero padding. The core action and its evidence scope are front-loaded, and the output-side effects close the sentence so the agent learns the call's footprint before invoking.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter analysis tool with no output schema, the description is nearly sufficient: it says what gets written and that a summary is returned. It stops short of describing what the summary contains or which of the many checks are fatal versus advisory, which would help an agent interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 80%, so the baseline is 3, and the description adds real value on top by expanding the accepted formats for 'path' (PDF, .docx, LaTeX file/folder/zip, Markdown, text) and by tying 'offline' to the reference-lookup behavior. It adds nothing for 'code' or 'lang', which remain schema-only.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description pairs a specific verb ('Fact-check') with a concrete resource ('a research paper') and enumerates the exact evidence dimensions checked (recomputed statistics, text-vs-table, reference retraction, figures, AI-writing residue) plus accepted input formats. It does not explicitly distinguish itself from the sibling render_report, leaving the division of labor to be inferred from 'Writes ... report.html to out_dir'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the reader can infer this is the entry point for validating a manuscript and that 'offline' suppresses network lookups. There is no explicit when-to-use / when-not-to-use guidance and no statement about how it relates to render_report or when to skip it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools