Skip to main content
Glama
@caelaralabs/mcp

@caelaralabs/mcp

caelara_agent_hallucination_diff

Compares a model answer with source text and flags unsupported sentences to detect hallucinations.

Instructions

[Agent Safety] Flag answer sentences unsupported by the source text Measured p50 11 ms, p95 15 ms (2026-11-21, benchmarks.caelaralabs.com). Calls https://agent.caelaralabs.com.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "source": "<source>",
      "answer": "<answer>"
    },
    "name": "caelara_agent_hallucination_diff"
  }
}

Input Schema

NameRequiredDescriptionDefault
answerYesModel answer to check
sourceYesSource / ground-truth text

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.1

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden; it does add real value by disclosing latency (p50 11 ms, p95 15 ms) and the backing endpoint. However, it says nothing about what a 'flag' consists of, input size limits, determinism, or error behavior, and the absence of an output schema makes that omission consequential.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Very short and front-loads the purpose before the operational metadata, which is good structure. The benchmark parenthetical is somewhat incidental for tool selection and the missing punctuation between the purpose and the latency sentence slightly hurts readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-string input tool this is nearly adequate, but with no output schema, it should describe what comes back: flagged sentence spans, confidence, indices, or counts. That return-shape gap is the main missing piece an agent needs to use the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with two clearly documented parameters ('answer' and 'source'), so the baseline is 3. The description adds no extra meaning about format, size limits, or how the two texts are aligned, so it does not exceed the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Flag') and object ('answer sentences unsupported by the source text'), which precisely describes a hallucination-detection diff. It does not differentiate itself from nearby siblings such as the citation extractor or output validator, so an agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The '[Agent Safety]' tag situates the tool in a category but gives no when-to-use or when-not-to-use guidance, no prerequisites, and no named alternative for related tasks like citation extraction. Scope and exclusions are left entirely to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools