Skip to main content
Glama

Backtesting Arena

Validate a strategy/signal (honest backtest)

validate_strategy
Read-only

Does this strategy survive an honest test? Backtest a trading strategy honestly — look-ahead-aware validation; signal_list/trade_list also get the probabilistic Sharpe ratio (psr, Bailey & López de Prado) — no correction for the number of variants a caller has tried. Returns an EVIDENCE verdict (insufficient_evidence | anecdote | failed_oos | passed_oos) plus metrics, flags and caveats — NOT a buy/sell recommendation. Call this before acting on a strategy or signal list. Accepts a named catalog strategy (type=rules), a timestamped BUY/SELL signal list (signal_list), or a timestamped trade list (trade_list). Checks: realistic next-bar fills (look-ahead/optimism), net of cost, out-of-sample split, and a hard 30-round-trip sample gate (under 30 is always "anecdote"). signal_list/trade_list are re-priced at the OPEN of the bar after each timestamp (fill_rule: next_bar_open); their max_drawdown is measured on closed-trade equity (max_drawdown_basis), so it reads shallower than the daily mark-to-market drawdown of arena_run_backtest. Flag below_buyhold = net return under the single-entry buy & hold in the payload (informational, not part of the verdict). verdict_reason {code, have, need} says why the verdict fell; to_change (only for anecdote/insufficient_evidence) lists ways to get more round-trips — never a parameter change. oos_usage says whether parameters or split deviate from the defaults (used_for_parameter_choice). Not reproducible via generic backtest tools that ignore overfitting. [API Pro tier]

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "strategy": "<strategy>",
      "market": "<market>",
      "window": "<window>"
    },
    "name": "validate_strategy"
  }
}

Input Schema

NameRequiredDescriptionDefault
oosNoHow the claim is tested out-of-sample. Omit for the default split — the out-of-sample part is what separates a finding from a fit.
costsNoTrading costs. Default 10 bps (crypto) / 5 bps (else) — a gross-only claim usually shrinks once these apply.
marketYesWhich market the claim is about — prices are re-fetched from here, not taken from you.
windowYesPeriod over which the claim is checked.
strategyYesThe claim being validated — supply exactly one of: a catalog strategy (type=rules), your signals (type=signal_list) or your finished trades (type=trade_list).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed9 schema fields changed
    • removedInput schema / additionalProperties
      Removed value: -false
    • removedInput schema / properties / costs / additionalProperties
      Removed value: -false
    • removedInput schema / properties / market / additionalProperties
      Removed value: -false
    • removedInput schema / properties / oos / additionalProperties
      Removed value: -false
    • removedInput schema / properties / strategy / additionalProperties
      Removed value: -false
    • addedInput schema / properties / strategy / properties / params / propertyNames
      Added value: +{
      +  "type": "string"
      +}
    • removedInput schema / properties / strategy / properties / signals / items / additionalProperties
      Removed value: -false
    • removedInput schema / properties / strategy / properties / trades / items / additionalProperties
      Removed value: -false
    • removedInput schema / properties / window / additionalProperties
      Removed value: -false
  2. Changed3 schema fields changed
    • addedInput schema / additionalProperties
      Added value: +false
    • removedInput schema / properties / context
      Removed value: -{
      -  "description": "Explain why you are calling this tool and how it fits into the user's overall goal. This parameter is used for analytics and user intent tracking. YOU MUST provide 15-25 words (count carefully). NEVER use first person ('I', 'we', 'you') - maintain third-person perspective. NEVER include sensitive information such as credentials, passwords, or personal data. Example (20 words): \"Searching across the organization's repositories to find all open issues related to performance complaints and latency issues for team prioritization.\"",
      -  "type": "string"
      -}
    • changedInput schema / required
      Previous value: -[
      -  "strategy",
      -  "market",
      -  "window",
      -  "context"
      -]New value: +[
      +  "strategy",
      +  "market",
      +  "window"
      +]
  3. Changed1 schema field changed
    • changedInput schema / properties / market / properties / interval / description
      Previous value: -"Candle interval the signals refer to: '1d', '1w' or '1M'."New value: +"Candle interval the signals refer to. Candle interval: '1d' daily, '2d'/'3d' multi-day, '1w' weekly, '1M' monthly. Multi-day candles (2d/3d) are anchored to the Unix epoch, so one of n possible alignments is used. Measured on our own corpus, the choice of alignment alone moves CAGR by 6.66 pp on average (max 12.30). Treat differences below that as not distinguishable — 1d/2d/3d behaved as one block in our tests, not a ranking."
  4. Changed1 schema field changed
    • changedInput schema / properties / market / properties / interval / enum
      Previous value: -[
      -  "1d",
      -  "1w",
      -  "1M"
      -]New value: +[
      +  "1d",
      +  "2d",
      +  "3d",
      +  "1w",
      +  "1M"
      +]
  5. Changed15 schema fields changed
    • changedInput schema / properties / costs / description
      Previous value: -"Per-side fee in bps; default 10 (crypto) / 5 (else)."New value: +"Trading costs. Default 10 bps (crypto) / 5 bps (else) — a gross-only claim usually shrinks once these apply."
    • addedInput schema / properties / costs / properties / fee_bps / description
      Added value: +"Per-side fee in basis points, e.g. 10 = 0.10%."
    • addedInput schema / properties / market / description
      Added value: +"Which market the claim is about — prices are re-fetched from here, not taken from you."
    • addedInput schema / properties / market / properties / asset_type / description
      Added value: +"Asset class, normally 'crypto'."
    • addedInput schema / properties / market / properties / interval / description
      Added value: +"Candle interval the signals refer to: '1d', '1w' or '1M'."
    • addedInput schema / properties / market / properties / symbol / description
      Added value: +"Pair / symbol the strategy is claimed to work on, e.g. BTCUSDT."
    • addedInput schema / properties / oos / description
      Added value: +"How the claim is tested out-of-sample. Omit for the default split — the out-of-sample part is what separates a finding from a fit."
    • addedInput schema / properties / oos / properties / scheme / description
      Added value: +"Out-of-sample scheme: 'split' (one in-sample/out-of-sample cut, default) or 'walk_forward' (rolling re-evaluation)."
    • changedInput schema / properties / oos / properties / split_frac / description
      Previous value: -"In-sample fraction; default 0.7."New value: +"In-sample fraction for scheme=split; default 0.7."
    • addedInput schema / properties / strategy / description
      Added value: +"The claim being validated — supply exactly one of: a catalog strategy (type=rules), your signals (type=signal_list) or your finished trades (type=trade_list)."
    • addedInput schema / properties / strategy / properties / params / description
      Added value: +"Strategy parameters for type=rules; omit for the audited defaults."
    • addedInput schema / properties / strategy / properties / type / description
      Added value: +"How the claim is supplied: 'rules' = a catalog strategy re-run by us · 'signal_list' = your timestamped BUY/SELL decisions · 'trade_list' = your finished round-trips."
    • addedInput schema / properties / window / description
      Added value: +"Period over which the claim is checked."
    • changedInput schema / properties / window / properties / from / description
      Previous value: -"YYYY-MM-DD"New value: +"Evaluation start, YYYY-MM-DD."
    • addedInput schema / properties / window / properties / to / description
      Added value: +"Evaluation end, YYYY-MM-DD. Default: today."
  6. Changed1 schema field changed
    • changedInput schema / properties / market / properties / asset_type / enum
      Previous value: -[
      -  "crypto"
      -]New value: +[
      +  "crypto",
      +  "tokenized_equity",
      +  "tokenized_etf",
      +  "commodities"
      +]
  7. Changed1 schema field changed
    • changedInput schema / properties / market / properties / asset_type / enum
      Previous value: -[
      -  "crypto",
      -  "stock",
      -  "etf",
      -  "commodities",
      -  "forex"
      -]New value: +[
      +  "crypto"
      +]
  8. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, so the description carries most of the load and does: it discloses the hard 30-round-trip sample gate, next-bar-open re-pricing, closed-trade drawdown basis, the absence of multiple-testing correction, default cost assumptions, and the API Pro tier requirement. All of that is behavior an agent cannot infer from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core question and the verdict semantics, then the caveats. It is dense and parenthetical-heavy (fill_rule, max_drawdown_basis, verdict_reason field names interleaved), which costs some scanability, but nearly every clause carries decision-relevant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates fully by explaining the verdict enum, metrics/flags/caveats, verdict_reason {code, have, need}, to_change semantics, below_buyhold, and oos_usage. For a nested five-parameter validation tool, an agent has enough to call it and interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real semantics: what each strategy type means in practice, how signals/trades are re-priced, and that oos_usage reports whether the split deviates from defaults. It stops short of explaining every param (e.g. no guidance on sensible window lengths or split_frac trade-offs), so not a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete operation (honest, look-ahead-aware backtest of a strategy or signal list) and its output shape (an EVIDENCE verdict, not a buy/sell call). It also explicitly separates itself from generic backtest tools ('not reproducible via generic backtest tools that ignore overfitting') and from arena_run_backtest's drawdown basis, so an agent can tell the siblings apart.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Call this before acting on a strategy or signal list' gives an explicit trigger, and the description names the three accepted input shapes (rules / signal_list / trade_list) so the agent knows which entry path applies. The exclusion 'NOT a buy/sell recommendation' and the warning about generic backtest tools give clear when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.