RefSentry — Citation Verifier MCP

A local-first Model Context Protocol server for citation verification and bibliography checking.

Check references and inspect whether a paper supports a factual claim, directly from Kilo or another MCP client. Bibliography tools use academic-refchecker; claim verification uses local DeBERTa inference, BM25 retrieval and exact evidence provenance.

Local inference, explicit network boundaries. Source downloads and RefChecker metadata lookups can contact external services. GROBID runs locally by default. See Network and privacy before processing sensitive material.

Install · Tools · Examples · Verdicts · Limitations

Citation hallucination is a growing problem in AI-assisted research. LLMs invent plausible-looking references that do not exist. This project provides a unified, local-first pipeline:

1. Load a supported source (local PDF, arXiv paper, or direct PDF URL)

2. Extract canonical PDF text

3. Retrieve relevant evidence chunks via BM25

4. Verify exact provenance (character-level offsets in the source)

5. Score claim–evidence pairs with a local DeBERTa NLI model

6. Gate strong contradiction candidates using topical overlap

7. Decide with a deterministic verdict policy

Features

Related MCP server: clearon-source-audit

Architecture

flowchart TD
    Client["MCP client"] --> Server["server.py · stdio"]
    subgraph V1["V1 · Bibliography"]
        direction TB
        Ref["academic-refchecker CLI"] --> Report["Report / citation summary"]
        Ref --> Grobid["GROBID · local by default"]
        Ref --> Metadata["External academic metadata"]
    end
    subgraph V2["V2 · Claim evidence"]
        direction TB
        Load["Source loading → PDF text extraction"] --> Chunks["Evidence chunking → BM25 retrieval"]
        Chunks --> NLI["Evidence windows → Local DeBERTa NLI"]
        NLI --> Guards["Exact provenance + contradiction relevance"]
        Guards --> Verdict["PASS / WARN / FAIL / ABSTAIN"]
    end
    Server -->|"3 bibliography tools"| V1
    Server -->|"2 claim tools"| V2
    classDef local fill:#eef2ff,stroke:#6366f1,color:#0f172a
    class Server,Load,Chunks,NLI,Guards,Verdict,Report,Grobid,Metadata local

V1 vs V2: citation_summary, verify_document, and verify_bibliography use the V1 RefChecker (external CLI). verify_claim and verify_claims use the V2 pipeline (local DeBERTa NLI + BM25 + provenance). See Architecture Guide.

Exact spans are checked before NLI scoring; the decision policy requires verified provenance. Coverage filters strong contradictions, including those used for conflict decisions, and does not independently validate entailments. V2 persists source text, chunks and verification records locally in SQLite.

Tools

V1 accepts source identifiers or files through RefChecker; “text” means a file, not arbitrary inline source text. V2 supports local PDFs, arXiv IDs/URLs and public direct PDF URLs; DOI full-text resolution is not implemented.

top_k is optional on the two V2 tools only: 1–20, default 5 unless CITATION_BM25_TOP_K overrides it. MCP discovery exposes claims as an array of generic objects; each item's claim and source strings are validated at runtime. Batches accept up to 100 items.

Requirements

No API keys or tokens required. All inference runs locally.

Installation (Windows)

Bootstrap requires Python >=3.13; the independently verified environment is Python 3.13.7 on Windows 11.

Bootstrap retains the selected interpreter and validates the Python version of newly created and reused .venv environments before installing packages. Without -Force, unsupported, broken or incomplete existing environments are preserved and rejected. Before explicit -Force recreation, confirm exclusive ownership, stop clients and back up the environment; see safe recovery.

# 1. Clone the repository
git clone https://github.com/LloydPhuc/citation-verifier-mcp.git
cd citation-verifier-mcp

# 2. Bootstrap environment (creates .venv, installs deps, downloads model)
.\scripts\bootstrap.ps1

# 3. Start GROBID (citation parsing service)
.\scripts\start.ps1

# 4. Verify everything is ready
.\scripts\doctor.ps1

# 5. Generate Kilo config snippet
.\scripts\print_kilo_config.ps1

Copy the JSON block from step 5 into your Kilo configuration (see Kilo Setup Guide).

Dependencies

The bootstrap script installs:

PyTorch CPU index: The CPU-only PyTorch build is installed from https://download.pytorch.org/whl/cpu. This index URL is not stored in pyproject.toml — users running manual installation must supply it.

Dependency commands used by bootstrap.ps1 (after creating .venv; the full bootstrap also prepares the model and checks imports):

.\.venv\Scripts\python.exe -m pip install torch==2.14.0+cpu --index-url https://download.pytorch.org/whl/cpu
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe -m pip install academic-refchecker==3.0.190

V1 RefChecker: academic-refchecker==3.0.190 is a pip-installable package. Bootstrap installs it separately when its executable is missing. The server invokes .venv\Scripts\academic-refchecker.exe as a subprocess.

Model: cross-encoder/nli-deberta-v3-small (~170 MB, cached locally after first download)

If model download stalls:

$env:HF_HUB_DISABLE_XET=1
.\scripts\bootstrap.ps1

GROBID and External Services

The start.ps1 script manages a local GROBID container via Docker Compose:

# Default port 8070
.\scripts\start.ps1

# Custom port
$env:GROBID_HOST_PORT=9070
$env:GROBID_URL="http://127.0.0.1:9070"
.\scripts\start.ps1

Security: The repository's Compose configuration binds GROBID to 127.0.0.1. Existing containers or a custom GROBID_URL may have different exposure. Set both port variables for a custom port and ensure Kilo inherits GROBID_URL after restarting it. See Kilo Setup.

Note: V2 verification (verify_claim, verify_claims) does not require GROBID. GROBID is only used by V1 tools (citation_summary, verify_document, verify_bibliography).

Kilo MCP Configuration

Generate the config snippet:

.\scripts\print_kilo_config.ps1

Named entry printed by the helper (illustrative paths; insert directly under mcp):

"citation-verifier": {
  "type": "local",
  "command": [
    "C:\\path\\to\\your\\repo\\.venv\\Scripts\\python.exe",
    "C:\\path\\to\\your\\repo\\server.py"
  ],
  "enabled": true,
  "timeout": 600000
}

Merge into ~/.config/kilo/kilo.jsonc (global) or <repo>/.kilo/kilo.jsonc (project-specific, overrides global). The helper separately prints the optional "citation-verifier_*": "allow" permission entry; merge it into the existing permission object if desired. It never edits your configuration.

Reload the client after config changes; in VS Code, use Developer: Reload Window. Environment changes require fully exiting the host and launching it from the configured shell; a window reload may retain old values.

Verify in Kilo: MCP panel shows citation-verifier as Connected with 5 tools.

Usage Examples

These calls use arXiv 2607.22693, the historical regression source. Response excerpts illustrate the public structure and recorded outcomes; they are not newly executed verification results. Model scores and external metadata can vary.

Single Claim Verification

{
  "claim": "The paper studies hallucinated and suspicious citations.",
  "source": "2607.22693"
}

Illustrative response excerpt. The quote is shortened for readability; offsets, page numbers and scores are omitted. This displayed excerpt cannot be used to validate exact provenance. A real response supplies the complete quote and exact offsets into canonical extracted text.

{
  "ok": true,
  "verdict": "PASS",
  "semantic_class": "SUPPORTED",
  "reason": "STRONG_ENTAILMENT_WITH_PROVENANCE",
  "evidence": {
    "quote": "…hallucinated and suspicious references have become a real and growing threat…",
    "provenance_verified": true
  }
}

Batch Verification (same-source reuse)

{
  "claims": [
    { "claim": "The paper studies hallucinated and suspicious citations.", "source": "2607.22693" },
    { "claim": "The paper does not discuss hallucinated citations.", "source": "2607.22693" }
  ]
}

{
  "total": 2,
  "source_loads": 1,
  "bm25_indexes_built": 1,
  "unique_sources": 1,
  "results": [
    { "index": 0, "verdict": "PASS", "semantic_class": "SUPPORTED", "reason": "STRONG_ENTAILMENT_WITH_PROVENANCE" },
    { "index": 1, "verdict": "FAIL", "semantic_class": "CONTRADICTED", "reason": "STRONG_CONTRADICTION" }
  ]
}

Citation Summary (V1 RefChecker)

{ "source": "2607.22693" }

{
  "ok": true,
  "total_references_processed": 24,
  "total_errors": 0,
  "total_warnings": 11,
  "total_unverified_references": 0,
  "total_information": 1,
  "citation_health": "Fair"
}

Verdicts

Key rules:

• PASS is forbidden unless provenance_verified == true.

• Numeric claims must match evidence numbers exactly.

• FAIL requires both strong contradiction and topical relevance — evidence below coverage 0.30 is excluded. This heuristic does not guarantee correct verdicts; synonym-based contradictions can be missed.

• Source/network/extraction failures → ABSTAIN, never FAIL.

• ABSTAIN means "could not verify," not "claim is false."

These are default thresholds. Strong-score cutoffs can be configured. See Verdicts Guide for the full decision policy.

Known Limitations

See Limitations for details.

Documentation

Contributing

Contributions are welcome. See CONTRIBUTING.md for guidelines.

Key requirements:

• All changes must include tests

• No secrets or private data in commits

• Provenance bypass is strictly forbidden

• Verdict policy changes require documented justification

Network and privacy

No API key is required for this server's local claim inference. Local inference does not establish end-to-end offline operation.

V2 stores PDFs, canonical text and verification records in local cache/SQLite. A local PDF with an already cached model can avoid source/model downloads, but the complete application is not guaranteed offline. Review external-service behavior before submitting confidential documents.

See SECURITY.md for vulnerability reporting. The policy describes network boundaries and the planned GitHub private reporting channel and its publication gate.

• GROBID binds to 127.0.0.1 by default (loopback only)

• The V2 source loader blocks non-public destinations and enforces download size limits, redirect limits and timeouts

• These V2 controls do not establish equivalent restrictions for RefChecker's external subprocess

License

This project is released under the MIT License (see LICENSE).

Third-party components have their own licenses:

• academic-refchecker — see its repository

• cross-encoder/nli-deberta-v3-small — model license (see Hugging Face)

• GROBID — Apache 2.0

• PyTorch, Transformers, etc. — their respective licenses

See THIRD_PARTY.md for the complete inventory.

Trace the citation. Inspect the evidence. Keep the research judgment yours.