Skip to main content
Glama

Draft Agent Response Groundedness & Hallucination Auditor

audit_response_factuality
Read-onlyIdempotent

Cross-reference factual claims, numbers, and citations in a draft agent response against retrieved source passages to flag unsupported hallucinations before delivery.

Instructions

Cross-references factual claims, numbers, and citations in a draft agent response against retrieved source passages to flag unsupported hallucinations before delivery. Use when verifying that a generated answer is 100% grounded in source evidence before returning it to the user; for scrubbing inbound tool outputs for prompt-injection and invisible Unicode, use sanitize_tool_output_stream instead. Stateless read-only edge call (max 32 KB, 60 RPM): accepts 'claim' ('context' defaults to 'standard'), supports '?demo=true' or L402 (25 sats), and returns HTTP 400 if invalid.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "claim": "<claim>",
      "context": "<context>"
    },
    "name": "audit_response_factuality"
  }
}

Input Schema

NameRequiredDescriptionDefault
claimYesDraft agent response or factual statement to verify for hallucinations.
contextYesAuthoritative source text or retrieved RAG passages that the claim must be grounded in.
strictnessNoEntailment threshold required for numbers and dates; defaults to 'exact_entailment' when omitted.

Output Schema

NameRequiredDescriptionDefault
verdictYesAudit outcome ('PASS', 'SUSPICIOUS', or 'FAIL').
verified_claimsYesAssertions directly entailed by the provided source context.
unverified_claimsYesHallucinated or unsupported assertions flagged for removal.
groundedness_scoreYesFactual support ratio from 0.0 (fabricated) to 1.0 (fully entailed).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.3.3

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish the safety profile (readOnly, idempotent, non-destructive, closed-world), and the description adds real operational context: a 32 KB payload cap, 60 RPM, a stateless edge call, L402 payment of 25 sats or '?demo=true', and an HTTP 400 on invalid input. The one flaw is the claim that "'context' defaults to 'standard'", which is misleading, because the schema marks context as required and no such default exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The tool's purpose is front-loaded in the first sentence, followed by usage, the sibling exclusion, and constraints. The final sentence crams four unrelated facts (size cap, rate limit, payment, error code) into one clause, but there's no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists, the description needn't explain return values, and it covers the operational envelope (limits, auth/payment, failure mode) that an agent needs before calling. The gaps are the undocumented strictness parameter behavior and the inaccurate 'context' default, both of which an agent would rely on.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each of the three parameters is already documented in the schema, and the baseline of 3 applies. The description restates parameter purpose loosely and introduces a default for 'context' that contradicts the required list, but it never touches the 'strictness' enum (exact_entailment / semantic_paraphrase / citations_only), which is where added semantics would have mattered most.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: cross-references factual claims, numbers, and citations in a draft response against retrieved source passages and flags hallucinations. That is far more concrete than the name alone. It only differentiates itself from one sibling (sanitize_tool_output_stream) and says nothing about the near-identical sibling verify_factual_claims, so an agent can't fully separate the two from this text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states an explicit trigger ('before returning it to the user' / 'before delivery') and names an alternative with the condition that selects it: use sanitize_tool_output_stream for prompt-injection and invisible Unicode scrubbing. However, the closest alternative, verify_factual_claims, is never mentioned, leaving the most likely routing ambiguity unresolved.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools