Skip to main content
Glama

Compare Strategy Runs

compare_strategy_runs
Read-only

Compare two backtest runs by metrics and provenance to reveal performance differences and confirm reproducibility.

Instructions

Compare canonical metrics and provenance using comparison-profile-v1.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
left_run_idYes
right_run_idYes

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, and the description's 'Compare' is consistent with a non-mutating operation. It adds context that the comparison covers canonical metrics and provenance under comparison-profile-v1, but it does not describe output behavior, errors, or side effects; the annotations cover the safety profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler, front-loading the action ('Compare') and the subject. It is appropriately brief for a simple tool, though the brevity contributes to the under-specification of usage and parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only comparison tool with two self-named parameters and an output schema, the description covers the core purpose. However, it leaves 'comparison-profile-v1' and 'canonical metrics' undefined and provides no guidance on how this differs from related run tools, so an agent may lack enough context to know when it is the right call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description says nothing about left_run_id or right_run_id. The parameter names and titles are self-explanatory, but the description does not compensate for the missing schema descriptions or explain what 'left' and 'right' mean in this comparison.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Compare'), names the resource ('strategy runs'), and specifies the object ('canonical metrics and provenance') plus the profile version. It is the only compare tool among the siblings, so an agent can likely distinguish it from get_run_result or inspect_strategy, though it does not explicitly call out those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to choose this over siblings such as get_run_result, render_strategy_report, or audit_independence. It implies comparison between two runs, but there is no 'use this when...' or 'for X use Y instead' statement, so the agent must infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.