Skip to main content
Glama

Test on the hold-out

run_held_out_test

The final out-of-sample check: runs a finished run's exact code once over its study's hold-out period. Call it at the end, on the run you would keep: the first held-out test per account and market is the one that counts, and a hold-out with too few trades for figures (a result needs 10) spends it all the same: each development run says how many trades the hold-out is expected to hold, in hold_out_estimate.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "run_id": "<run_id>"
    },
    "name": "run_held_out_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
run_idYesA run of a study with a hold-out, which ran.

Output Schema

NameRequiredDescriptionDefault
nextNo
basisNo
gradeNo
linesNo
marketNo
reasonNo
run_idNo
statusYes
windowNo
messageNo
metricsNo
noticesNo
versionNo
headlineNo
held_outNo
settingsNo
study_idNo
warningsNo
retryableNo
cost_tableNoEvery cost the run charged: each component (commission, slippage, spread) with its amount, unit and basis, what it comes to per round trip, when it is taken, and whether it is yours (caller) or the market's default; the round trip summed by unit (pips on a currency pair, the costs stated in pips). Costs you send replace the market's defaults side by side: a commission replaces the default commission, a slippage or a spread replaces the default slippage, a side you leave out is charged at the market's default, and nothing is added on top.
disclaimerNoThe hypothetical-performance line, on a result (a run, or a study's runs): the same text the app shows with every result.
look_aheadNo
trade_listNoThe run's trades in the order they were made, a page at a time: when each entered and left (a day on daily bars; a time on the `clock` on shorter ones; "open at the end" for a trade still open when the data ended), its side, whether it won or lost after costs, and which exit ended it (a word: stop, exit rule, target, time limit…). No prices, levels, sizes or returns.
buy_and_holdNo
replayed_fromNo
grade_withheldNo
runs_left_todayNo
hold_out_estimateNo
poll_after_secondsNo
stated_rule_breaksNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changed
    • changedOutput schema / properties / cost_table / properties / components / items / properties / unit / enum
      Previous value: -[
      -  "percent",
      -  "bps",
      -  "usd_per_share",
      -  "usd_per_ounce",
      -  "pips",
      -  "usd_per_order"
      -]New value: +[
      +  "percent",
      +  "bps",
      +  "usd_per_share",
      +  "usd_per_ounce",
      +  "pips",
      +  "usd_per_order",
      +  "usd_per_lot"
      +]
    • addedOutput schema / properties / settings / properties / costs_charged / properties / value / items / properties / lot_ounces
      Added value: +{
      +  "type": "number"
      +}
    • changedOutput schema / properties / settings / properties / costs_charged / properties / value / items / properties / unit / enum
      Previous value: -[
      -  "percent",
      -  "bps",
      -  "usd_per_share",
      -  "usd_per_ounce",
      -  "pips",
      -  "usd_per_order"
      -]New value: +[
      +  "percent",
      +  "bps",
      +  "usd_per_share",
      +  "usd_per_ounce",
      +  "pips",
      +  "usd_per_order",
      +  "usd_per_lot"
      +]
  2. Changed3 schema fields changed
    • addedOutput schema / properties / cost_table
      Added value: +{
      +  "additionalProperties": false,
      +  "description": "Every cost the run charged: each component (commission, slippage, spread) with its amount, unit and basis, what it comes to per round trip, when it is taken, and whether it is yours (caller) or the market's default; the round trip summed by unit (pips on a currency pair, the costs stated in pips). Costs you send replace the market's defaults side by side: a commission replaces the default commission, a slippage or a spread replaces the default slippage, a side you leave out is charged at the market's default, and nothing is added on top.",
      +  "properties": {
      +    "components": {
      +      "items": {
      +        "additionalProperties": false,
      +        "properties": {
      +          "amount": {
      +            "type": "number"
      +          },
      +          "basis": {
      +            "enum": [
      +              "per_side",
      +              "round_trip"
      +            ],
      +            "type": "string"
      +          },
      +          "charged": {
      +            "description": "When and on what the engine takes it.",
      +            "type": "string"
      +          },
      +          "component": {
      +            "enum": [
      +              "commission",
      +              "slippage",
      +              "spread"
      +            ],
      +            "type": "string"
      +          },
      +          "origin": {
      +            "description": "caller: your costs; default: the market's default.",
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          },
      +          "per_round_trip": {
      +            "additionalProperties": false,
      +            "properties": {
      +              "amount": {
      +                "type": "number"
      +              },
      +              "unit": {
      +                "enum": [
      +                  "bps",
      +                  "pips",
      +                  "usd",
      +                  "usd_per_share",
      +                  "usd_per_ounce"
      +                ],
      +                "type": "string"
      +              }
      +            },
      +            "required": [
      +              "amount",
      +              "unit"
      +            ],
      +            "type": "object"
      +          },
      +          "unit": {
      +            "enum": [
      +              "percent",
      +              "bps",
      +              "usd_per_share",
      +              "usd_per_ounce",
      +              "pips",
      +              "usd_per_order"
      +            ],
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "component",
      +          "amount",
      +          "unit",
      +          "basis",
      +          "per_round_trip",
      +          "charged",
      +          "origin"
      +        ],
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    "defaults_replaced": {
      +      "description": "The market defaults your costs replaced: none of them was charged, and nothing was added to your costs.",
      +      "items": {
      +        "enum": [
      +          "commission",
      +          "slippage"
      +        ],
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "round_trip_total": {
      +      "additionalProperties": false,
      +      "properties": {
      +        "bps_of_value": {
      +          "type": "number"
      +        },
      +        "pips": {
      +          "type": "number"
      +        },
      +        "usd": {
      +          "type": "number"
      +        },
      +        "usd_per_ounce": {
      +          "type": "number"
      +        },
      +        "usd_per_share": {
      +          "type": "number"
      +        }
      +      },
      +      "required": [
      +        "bps_of_value"
      +      ],
      +      "type": "object"
      +    }
      +  },
      +  "required": [
      +    "components",
      +    "defaults_replaced",
      +    "round_trip_total"
      +  ],
      +  "type": "object"
      +}
    • addedOutput schema / properties / market / properties / derived_only / description
      Added value: +"True when the market-data licence lets the account see only results derived from the prices (figures, never the prices or bars themselves). It says what may be shown, not how the quote is made."
    • addedOutput schema / properties / window / properties / gaps
      Added value: +{
      +  "items": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "from": {
      +        "type": "string"
      +      },
      +      "to": {
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "from",
      +      "to"
      +    ],
      +    "type": "object"
      +  },
      +  "type": "array"
      +}
  3. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false and idempotentHint=false; the description goes well beyond by disclosing that the hold-out is consumed permanently, that results under 10 trades still burn the attempt, and that `hold_out_estimate` from a development run can be checked beforehand.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and free of filler, though the second sentence is a dense colon-and-dash chain that takes a second pass to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values need no explaining; the description covers everything an agent needs, including the one-shot nature, the minimum-trade threshold, and the pre-check via `hold_out_estimate`.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is documented there, but the description adds real selection guidance: the run must be a finished one with a hold-out, and it should be the run you intend to keep.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('runs a finished run's exact code once over its study's hold-out period') and immediately frames it as the final out-of-sample check, which separates it from run_backtest's development-phase role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to call ('at the end, on the run you would keep') and adds the crucial constraint that only the first held-out test per account and market counts, which governs the decision of when to invoke it versus iterating further.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources