This server provides a hallucination detection service that verifies a draft text against a source of truth, identifying unsupported factual claims.
Core capability: Submit a source (trusted reference) and a draft (text to check) to receive a structured JSON report ({passed, failures, uncertain, mode}) with a pass/fail verdict and details on every flagged claim — including claim, reason, source_fact_checked, category, and severity.
Additional options:
Two modes: sampling (default, uses the host model via MCP — no API key needed) or api (uses a pinned Anthropic model directly when ANTHROPIC_API_KEY is set, benchmarked at 97.9% precision / 98.5% F1).
Reliability mode: Set consistency to N (e.g., 3) to run verification multiple times, keeping only majority-confirmed failures and flagging unstable ones as uncertain.
Pipeline-friendly: The machine-readable JSON output is suitable for CI/CD, content review, or document generation workflows.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mcp-verify Verify this text against the provided source."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A benchmarked hallucination detector. Give it a SOURCE (the source of truth) and a
DRAFT; it returns the specific claims the source doesn't support, each with
{claim, reason, source_fact_checked, category, severity}.
Benchmarked: 106 labeled cases across 4 domains + an adversarial red-team tranche. Over 3 independent runs: precision 97.9% [96.6, 99.1], recall 99.1%, F1 98.5% (95% CIs) — with every residual failure adjudicated and documented. See BENCHMARK.md.
from mcp_verify import build_default_client, verify
client = build_default_client() # reads ANTHROPIC_API_KEY
report = verify(client, source="...source of truth...", draft="...text to check...")
for f in report.failures:
print(f.severity, "|", f.claim, "—", f.reason)
# Reliability mode: run N times, keep only majority-confirmed claims;
# unstable ones land in report.uncertain instead of report.failures.
report = verify(client, source="...", draft="...", consistency=3)One tool, verify(source, draft, consistency=1), over stdio:
pip install mcp-verify # or: pip install -e . from a checkout
claude mcp add verify -- mcp-verifyTwo modes, auto-selected (MCP_VERIFY_MODE=api|sampling overrides):
sampling (default, no key): the server asks the HOST's model to do the verification via MCP sampling — no ANTHROPIC_API_KEY, no extra cost.
api: if ANTHROPIC_API_KEY is set in the server's environment, it calls
the pinned benchmark model directly — the exact path BENCHMARK.md measures.
claude mcp add verify -e ANTHROPIC_API_KEY=sk-... -- mcp-verify
Both return the same JSON: {"passed", "failures": [...], "uncertain": [...], "mode"}.
Related MCP server: Fact-Checker MCP Server
The eval suite (106 labeled cases across 4 domains plus a red-team tranche, a 15-type hallucination taxonomy, and a confusion matrix) travels with the detector. See BENCHMARK.md.
# Offline checks (no API key, no cost):
bash check.sh # lint + full test suite
python -m eval.analysis --sample # failure analysis on a sample report
# Live eval (needs ANTHROPIC_API_KEY):
python -m eval.run # full suite
python -m eval.run --smoke # 5 representative cases
python -m eval.run --judge # LLM-judge matcher (meaning, not tokens)
python -m eval.ablation # model/thinking/token-budget ablationCheck a DRAFT text against a SOURCE (the source of truth) and report every factual claim the source does not support.
Args: source: The trusted reference text — the source of truth to check against. draft: The text to verify (e.g. a summary, answer, or generated document).
Returns a JSON string: {"passed": bool, "failures": [{"claim", "reason", "source_fact_checked", "category", "severity"}], "mode": "api"|"sampling"}. passed is true only when no unsupported claims were found.
| Name | Required | Description | Default |
|---|---|---|---|
| draft | Yes | ||
| source | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the return format and key behaviors (e.g., passed only true when no failures, mode field), but does not mention side effects, auth needs, or rate limits. Since annotations are absent, the description carries the full burden and does so well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a brief opening sentence states the purpose, followed by an 'Args' section and a 'Returns' section. Every sentence adds value, and the important information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (two string parameters), the description fully covers all necessary information: purpose, parameter semantics, and return value format. No additional context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only provides names and types for the two string parameters. The description compensates fully by explaining that 'source' is the trusted reference text and 'draft' is the text to verify, adding critical semantic meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Check' and the resources 'DRAFT text' and 'SOURCE', and specifies the outcome of reporting unsupported factual claims. It leaves no ambiguity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool (when a draft needs fact-checking against a source), but does not explicitly mention when not to use it or alternatives. Since there are no sibling tools, this is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0verifyScored across 1 tool
With only one tool, there is no risk of ambiguity or confusion between tools. The 'verify' tool has a clear, singular purpose.
A single tool cannot have naming inconsistencies. The name 'verify' is a clear verb that directly describes its action.
The single tool is at the low end of the typical range. While the server is highly focused, a single tool feels thin for a standalone server, as users might expect additional related functionality.
The server provides a single verification operation, which is complete for that narrow task, but lacks any supporting tools (e.g., source management, draft editing) that would make it a comprehensive verification service.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Security scanner for MCP servers. Detect vulnerabilities, prompt injection, and tool poisoning.
MCP server for AI dialogue using various LLM models via AceDataCloud
