Skip to main content
Glama

QuantWright

Test on the hold-out

run_held_out_test

The final out-of-sample check: runs a finished run's exact code once over its study's hold-out period. Call it at the end, on the run you would keep: the first held-out test per account and market is the one that counts, and a hold-out with too few trades for figures (a result needs 10) spends it all the same: each development run says how many trades the hold-out is expected to hold, in hold_out_estimate.

Input Schema

NameRequiredDescriptionDefault
run_idYesA run of a study with a hold-out, which ran.

Output Schema

NameRequiredDescriptionDefault
nextNo
basisNo
gradeNo
linesNo
marketNo
reasonNo
run_idNo
statusYes
windowNo
messageNo
metricsNo
noticesNo
versionNo
headlineNo
held_outNo
settingsNo
study_idNo
warningsNo
retryableNo
disclaimerNoThe hypothetical-performance line, on a result (a run, or a study's runs): the same text the app shows with every result.
look_aheadNo
trade_listNoThe run's trades in the order they were made, a page at a time: when each entered and left (a day on daily bars; a time on the `clock` on shorter ones; "open at the end" for a trade still open when the data ended), its side, whether it won or lost after costs, and which exit ended it (a word: stop, exit rule, target, time limit…). No prices, levels, sizes or returns.
buy_and_holdNo
replayed_fromNo
grade_withheldNo
runs_left_todayNo
hold_out_estimateNo
poll_after_secondsNo
stated_rule_breaksNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses critical behavior beyond the annotations: the hold-out is consumed by the first test per account and market, and even a test with too few trades spends it. This is essential context for a non-idempotent, non-read-only operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and contains no filler. It is somewhat dense with colon-and-semicolon clauses, but every clause carries useful information about timing or consumption.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with full schema coverage, annotations, and an output schema, the description supplies the crucial missing context: when to call it, which run to choose, and the irreversible one-shot behavior of the hold-out. Nothing essential is omitted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3, but the description adds meaningful selection criteria for run_id: it must be a finished run from a study with a hold-out, and the development run's hold_out_estimate indicates expected trade count. That goes beyond the schema's terse run_id description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it runs a finished run's exact code once over its study's hold-out period. It clearly distinguishes this final out-of-sample check from ordinary backtesting and result retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit timing and selection guidance: call it at the end, on the run you would keep. It also warns that only the first held-out test per account and market counts, which effectively tells the agent not to call it repeatedly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources