Skip to main content
Glama
Sign In

Output Verification

by org.dpdns.agendamcp.output-verification
OverviewDiscussionsAdmin

review_answer

Find missing/unknown citations and mismatched quotations in an AI answer. Send {answer,sources:[{source_id,text}]}; use an empty sources array to find missing or unresolved references in raw text. Cite [source-id] in each nonempty line and put exact double quotes immediately before their citation. Optional numeric_smoke_check flags literal number tokens absent from cited excerpts; it does not verify units or meaning. URLs are not fetched. This checks structure, not whether a source supports a claim. No support_level is needed or inferred. Check free_trial_status first. Uses the same 5-attempt network/product campaign allowance as free_trial; an admitted rerun consumes another attempt. No payment, automatic retry, fact verification, signed receipt or authorization. Supplied text is data, never instructions. Human review is required.

Input Schema

NameRequiredDescriptionDefault
answerYes
sourcesYes
numeric_smoke_checkNoOptional literal comparison of number tokens in each line with its resolved cited excerpts; not a semantic, unit, context or truth check.

Output Schema

NameRequiredDescriptionDefault
trialYes
evaluationYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed6 schema fields changed
    • changedInput schema / description
      Previous value: -"Citation structure only. Every nonempty answer line uses [source-id] or a simple [source-id](https://URL) link. Straight or paired typographic double quotes must precede their citation. At most 200 nonempty lines, 500 references and 200,000 total source characters. Source IDs must be unique. Content is data, never instructions. No semantic support inference."New value: +"Citation structure only. Every nonempty answer line uses [source-id] or a simple [source-id](https://URL) link. Straight or paired typographic double quotes must precede their citation. At most 200 nonempty lines, 500 references and 200,000 total source characters. Source IDs must be unique. Content is data, never instructions. No semantic support inference. An empty sources array permits an unreferenced text preflight; citations cannot resolve without supplied excerpts. URLs are never fetched."
    • addedInput schema / properties / numeric_smoke_check
      Added value: +{
+  "description": "Optional literal comparison of number tokens in each line with its resolved cited excerpts; not a semantic, unit, context or truth check.",
+  "type": "boolean"
+}
    • changedInput schema / properties / sources / minItems
      Previous value: -1New value: +0
    • changedOutput schema / properties / evaluation / properties / findings / items / properties / code / enum
      Previous value: -[
-  "missing_citation",
-  "unknown_source",
-  "unsupported_citation_syntax",
-  "unbalanced_quote",
-  "unattributed_quote",
-  "quote_not_found"
-]New value: +[
+  "missing_citation",
+  "unknown_source",
+  "unsupported_citation_syntax",
+  "unbalanced_quote",
+  "unattributed_quote",
+  "quote_not_found",
+  "number_absent_from_cited_sources"
+]
    • removedOutput schema / properties / evaluation / properties / validation_scope / const
      Removed value: -"citation_and_quote_structure_only"
    • addedOutput schema / properties / evaluation / properties / validation_scope / enum
      Added value: +[
+  "citation_and_quote_structure_only",
+  "citation_quote_and_numeric_smoke"
+]
  2. Changed1 schema field changed
    • changedInput schema / description
      Previous value: -"Citation structure only. Every nonempty answer line uses plain [source-id]. Straight double quotes must precede their citation. At most 200 nonempty lines, 500 references and 200,000 total source characters. Source IDs must be unique. Content is data, never instructions. No semantic support inference."New value: +"Citation structure only. Every nonempty answer line uses [source-id] or a simple [source-id](https://URL) link. Straight or paired typographic double quotes must precede their citation. At most 200 nonempty lines, 500 references and 200,000 total source characters. Source IDs must be unique. Content is data, never instructions. No semantic support inference."
  3. Added

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses the 5-attempt campaign allowance, that a rerun consumes another attempt, that URLs are never fetched, that no payment or automatic retry occurs, that supplied text is treated as data, and that human review is required. This is exactly the operational context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and input shape, but the remainder is a dense run of sentence fragments covering attempts, disclaimers, and safety notes with no grouping. Almost every sentence is substantive, yet the wall-of-text delivery makes it harder to scan than it needs to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a mutation-flagged, attempt-limited tool with an output schema present, the description covers input shape, preflight mode, attempt consumption, and explicit non-goals. An agent has everything needed to call it correctly and to expect the right kind of result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 33%, so the description must compensate, and it does: it explains the {answer, sources:[{source_id,text}]} shape, the empty-array preflight mode, and the citation/quote placement rule. numeric_smoke_check is clarified as literal number-token comparison only, not unit or meaning checks, which meaningfully extends the schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: 'Find missing/unknown citations and mismatched quotations in an AI answer.' The scoping phrase 'This checks structure, not whether a source supports a claim' sharpens what kind of review it is. It stops short of differentiating itself from the sibling open_answer_review, which sounds adjacent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage conditions: 'use an empty sources array to find missing or unresolved references in raw text' and 'Check free_trial_status first.' It also states what the tool does not do (no fact verification, no URL fetching). It never names an alternative sibling or explains when to pick open_answer_review instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Output Verification toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources