Skip to main content
Glama
GodPrompt MCP Server

GodPrompt MCP Server

Compare paired agent evaluation runs

compare_evaluation_runs
Read-onlyIdempotent

Compares caller-reported baseline and candidate outcomes on matching case IDs, identifies coverage exclusions, and warns about low sample sizes.

Instructions

Compare bounded caller-reported outcomes on exact matching case IDs, with coverage exclusions and low-sample warnings. No API calls, statistical significance claims or stored results.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "baseline": "<baseline>",
      "candidate": "<candidate>"
    },
    "name": "compare_evaluation_runs"
  }
}

Input Schema

NameRequiredDescriptionDefault
baselineYes
candidateYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.4.2

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, closed-world, non-destructive behavior, yet the description adds real substance: coverage exclusions occur, low-sample warnings are emitted, and the tool makes no statistical significance claims and stores nothing. Those disclaimers meaningfully shape how an agent should interpret output. The main gap is that it does not sketch the shape of the returned comparison.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences with the purpose front-loaded and the caveats second. Every clause carries information, including the disclaimers, and nothing is repeated from the title or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should say what a comparison produces (per-metric deltas, pass-rate change, excluded-case handling), and it does not. It is adequate on input expectations and honest about limitations, but an agent cannot predict the return structure. This is a noticeable gap for a two-parameter analytical tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry the load, and it only partly does: 'exact matching case IDs' explains how baseline and candidate are joined and 'bounded' hints at the 200-item cap. It says nothing about the nested passed, cost_usd, or duration_ms fields, so the payload semantics of each entry remain undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: compare caller-reported outcomes between two runs on matching case IDs. It also scopes the input as caller-supplied and bounded, which distinguishes it from any sibling that computes or fetches data. It stops short of naming a contrasting sibling, but none of the listed siblings covers comparison, so ambiguity is low.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies when the tool applies (you already hold two result sets and want a paired comparison) but never states prerequisites or alternatives explicitly. The 'No API calls... or stored results' clause reads as behavioral boundaries rather than routing guidance. An agent can infer the use case but gets no explicit when-not condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.