Skip to main content
Glama

Compare two runs

compare_runs
Read-onlyIdempotent

Use this to compare the named rate of two runs on the questions and engines both measured. It reports Moved up or Moved down only where the intervals separate, and refuses to compare different method versions.

Input Schema

NameRequiredDescriptionDefault
run_idYesA run_id from list_runs.
other_run_idYesA run_id from list_runs.

Output Schema

NameRequiredDescriptionDefault
leftNo
rightNo
changeYes
sampleNoTrue on every answer about the fictional sample.
comparableYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a safe, idempotent, read-only operation, so the bar is lower and the description earns credit for adding real behavior: it only counts questions/engines measured in both runs, it reports directional movement only where confidence intervals separate, and it refuses cross-method-version comparisons. These are meaningful rules an agent cannot infer from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, both front-loaded and waste-free: the first states purpose and scope, the second states the reporting rule and the refusal condition. Nothing is repeated from the structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations covering safety and an output schema covering return values, the description only needed to supply comparison semantics and failure modes, which it does. The one mild gap is the unexplained jargon 'named rate', which an agent may not resolve from the remaining fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters documented as 'A run_id from list_runs'. The description implies the two-argument comparison but adds no format, ordering, or role detail (e.g. which run is the baseline) beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (compare) and resource (two runs) plus the precise scope being compared: the named rate on questions and engines both measured. It is clearly distinguishable from siblings like get_results or get_history, though it never names an alternative to reinforce that distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It opens with an explicit 'Use this to compare...' and supplies the operational context (comparing runs on the shared question/engine set). It also states a precondition/refusal case (different method versions cannot be compared), but gives no direct when-not or alternative-tool guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources