Agent Output Guard MCP

output_consistency_score

Calculate overall consistency score for agent output by analyzing internal logic, format consistency, and reliability indicators to provide a comprehensive reliability assessment.

Instructions

Calculate overall consistency score for agent output including internal logic, format consistency, and reliability indicators. Returns comprehensive reliability assessment.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "output": "<output>"
    },
    "name": "output_consistency_score"
  }
}

Input Schema

Name	Required	Description
`output`	Yes	Complete output from an agent (text, data, or structured response)
`context`	No	Context or prompt that generated this output
`source_agent`	No	Agent identifier for tracking reliability over time
`expected_format`	No	Expected structure/format of the output
`historical_outputs`	No	Previous outputs from same agent for pattern analysis

Schema Changelog

Changes observed during successful MCP inspections.

First observedv1.0.4Aug 4, 2026

TDQS

B3.4/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It says the tool 'returns comprehensive reliability assessment' and covers 'internal logic, format consistency, and reliability indicators' — which gives some insight into what the output contains. However, it doesn't disclose what input is required beyond the schema, whether it's a read-only analysis operation, what happens with missing context fields, or how the score is calibrated/normalized. For a scoring tool with no annotations, more behavioral context would help.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, compact sentence that efficiently conveys the tool's purpose and its key scoring dimensions. It's front-loaded with the core action ('calculate overall consistency score') and adds relevant detail about what dimensions are covered. No wasted words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a moderately complex scoring tool with 5 parameters, one nested object (historical_outputs), and no output schema to clarify the return value. The description explains what's being scored but doesn't explain the score format, scale, or how to interpret the 'comprehensive reliability assessment' output. Given the complexity (nested objects, multiple optional inputs) and lack of an output schema, the description could benefit from explaining how the various parameters factor into the score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning all 5 parameters have descriptions in the schema itself. The description adds the meaning that 'output' is the primary input and frames the tool's purpose around scoring it. However, the description doesn't add parameter-level semantics beyond the schema — it doesn't clarify what 'context' adds to scoring, how historical_outputs weighting works, or what expected_format contributes. Baseline 3 is appropriate since the schema is fully descriptive and the description doesn't materially deepen parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('calculate') with a clear resource ('overall consistency score for agent output') and lists the score dimensions (internal logic, format consistency, reliability). It's clear about what the tool does. However, it doesn't explicitly distinguish from siblings like detect_hallucination_markers or cross_reference_check, though the mention of 'overall consistency' hints at the holistic purpose versus the more targeted sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is for holistic output assessment (calculating an overall score), which is somewhat distinct from the sibling tools (verify_json_schema is structural, detect_hallucination_markers is specific, cross_reference_check is relational). However, there's no explicit 'when to use this vs alternatives' guidance or exclusion criteria. The usage context is only implied through the 'overall' framing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server