Skip to main content
Glama

Compare two passages for factual agreement

jev_compare

Compare two passages to judge whether they state the same fact, contradict, or differ, returning probability, confidence, and optional per-aspect verdicts for source checks.

Instructions

Judge the relation between two passages with TypeSafe Jev: same_fact, contradicts, or different_facts, with the full probability distribution, confidence, and an auto-versus-review decision. Optionally supply aspects (price, date, method, …) and each gets an independent per-aspect judgment in the same single request. Use for source reconciliation, changelog-vs-code drift, or merge sanity checks. The request supplies no evidence beyond the two passages, so a same_fact verdict means they agree with each other, not that they are true.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
aspectsNoNamed aspects to judge independently (e.g. 'price', 'launch date'). Each tests one property.
purposeNoWhat this comparison is for; helps disambiguate overlap.
passage_aYesFirst passage. Rejected above 20,000 characters.
passage_bYesSecond passage. Rejected above 20,000 characters.
auto_acceptNoMinimum top probability for auto. Default 0.85.
minimum_marginNoMinimum winner-to-runner-up gap for auto. Default 0.5.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.10.1
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
  2. Addedv0.4.0

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full load and does well: it discloses the returned verdict labels, the full probability distribution, a confidence value, and the auto-versus-review gating. It also flags the important epistemic caveat that same_fact means the passages agree with each other, not that they are true.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with the core action and verdicts, then adds optional aspects, use cases, and a closing caveat. Each sentence carries information, though the middle clause about auto-versus-review is dense and the sentences run long.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description fills most of the gap by describing the response shape (distribution, confidence, auto/review decision) and the per-aspect behavior. Only sibling disambiguation is left unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter including the auto_accept/minimum_margin defaults. The description adds only modest meaning, mainly that aspects yield independent per-aspect judgments, which is the baseline-plus level.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource (judge the relation between two passages) and enumerates the verdict space (same_fact, contradicts, different_facts), so the agent knows exactly what this produces. It does not differentiate from the very similar sibling jev_verify, which is the main missing piece.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It supplies concrete usage contexts (source reconciliation, changelog-vs-code drift, merge sanity checks), which is clearly better than no guidance. However, it never states when NOT to use it or names an alternative sibling such as jev_verify, so the agent must infer the boundary itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.