Skip to main content
Glama

Compare Backtests

tt_backtest_compare
Read-onlyIdempotent

Put two or three finished backtests side by side. FREE — reads runs already paid for, runs no engine, charges nothing.

This is the second half of research. One run tells you what happened; the comparison tells you whether the change you made did anything.

READ attribution BEFORE ANY NUMBER, and say what it says. A delta is attributable to a cause only when EXACTLY ONE run input differs — the window, the candle frequency, the type, the expiry, the fill price or the strategy. If more than one moved, this is two different experiments and no difference can be pinned on any of them. If none moved, the gap is the engine's own run-to-run variation, not a result.

THERE IS NO WINNER FIELD, deliberately. "Which is better" over a short window is how people talk themselves into noise. You get deltas, a paired daily-difference test, and the job of drawing the conclusion in the open.

daily_test.verdict is not a footnote — quote it alongside the delta. A bigger number over a few dozen days is usually chance, and the test says so.

Metrics in not_compared were withheld on one side for being unsupportable; subtracting them would launder them back in. Never do it.

Args: job_ids: 2-3 ids; the first is the baseline. sections: extra sections per run.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "job_ids": "<job_ids>"
    },
    "name": "tt_backtest_compare"
  }
}

Input Schema

NameRequiredDescriptionDefault
job_idsYesTwo or three run ids to compare. The FIRST is the baseline; the rest are variants measured against it.
sectionsNoExtra sections to pull for each run. headline, meta and series are always fetched.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive, closed-world, but the description adds genuinely non-redundant behavior: no engine runs and no charge, the exact-one-differing-input attribution rule, the deliberate absence of a winner field, the significance test in daily_test.verdict, and the warning that not_compared metrics were withheld as unsupportable. These are interpretation-critical traits an agent could not derive from the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence and each subsequent block carries real guidance (attribution, no winner, verdict quoting, withheld metrics). It is longer than typical and includes some editorializing ("how people talk themselves into noise") plus an Args section that duplicates the schema, but no paragraph is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so partially, naming attribution, deltas, daily_test.verdict, and not_compared rather than enumerating the full payload. For a 2-parameter read tool with clean annotations this is enough to call it safely, though the exact response shape remains partly implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters fully. The description's Args block ("2-3 ids; the first is the baseline", "extra sections per run") largely restates the schema, adding no new syntax, format, or edge-case detail, which lands at the baseline for high-coverage schemas.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Put two or three finished backtests side by side" states a specific verb and resource, and the following line ("One run tells you what happened; the comparison tells you whether the change you made did anything") draws an explicit boundary against the single-run sibling. An agent can tell immediately this is a multi-run comparison, not a fetch of one result.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It positions the tool in a workflow ("the second half of research"), requires finished runs, and states the 2-3 id / baseline-first constraint. It does not name a sibling tool to route to for a single run, leaving that inference to the reader, which keeps it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources