Skip to main content
Glama
Sign In
exp-agent

exp-agent

by lfy20040505666
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

compare_runs

Ranks multiple training runs by a selected final metric (PSNR, SSIM, or LPIPS) and reports the top run, treating lower LPIPS as better.

Instructions

Rank several runs by one final metric and say which is best. Knows that lower lpips is better.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "runs": "<runs>",
      "metric": "<metric>"
    },
    "name": "compare_runs"
  }
}

Input Schema

NameRequiredDescriptionDefault
runsYesRun names to compare.
metricYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does disclose a genuinely useful behavioral fact that lower lpips is better, which an agent could not infer from the schema. However, it never confirms the operation is read-only, and it does not describe the shape of the result (ranking order, how the best run is marked, tie handling).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core action and followed by the one non-obvious domain fact. Nothing is padded or repeated from the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a small read-style tool with two required parameters and no output schema. For that complexity the description covers the action and the key metric-direction nuance, though with no output schema it leaves the agent guessing at the return format. That is the one real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%: 'runs' is documented in the schema and the 'metric' parameter has an enum, but its semantics are undocumented there. The description compensates for the metric parameter by explaining that lower lpips is better, which is meaningful added value, but it says nothing about how multiple runs are supplied or what happens with one run.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: rank runs by a final metric and identify the best. This is clearly distinct from the read-oriented siblings (get_final_metrics, get_metric_curve) because it produces a ranking verdict, though it never explicitly names or contrasts those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: you gather that it is for comparing multiple runs. There is no explicit "use this instead of get_final_metrics when..." guidance, no mention of prerequisites such as needing completed runs, and no exclusion cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools