Canli Validation

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnly=false, destructive=false, openWorld=true, so the description carries most of the burden. It adds real behavioral context: each result ships with 'its receipt, plus fix_next', and it explicitly warns that a deflated Sharpe or overfitting probability above/below any threshold 'is not admission to anything and is not a forecast' – an important interpretive caveat. It stops short of noting that file arguments are read from the local machine (left to the schema).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, which is good, but the prose is dense and run-on, with parenthetical jargon like 'luck trials with Lo's correction without' that is hard to parse. No sentence is pure padding, yet the readability cost is real for a description this compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not enumerate return fields, and it still summarizes the payload (deflated Sharpe, minimum track record, overfitting, OOS decay, receipts, fix_next). For an 11-parameter statistical tool this is largely complete; the main omission is any hint about compute cost or runtime for large variants files.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 and the schema already documents all 11 parameters. The description adds only emphasis – 'Send every variant tried (variants_file) so trials are counted' – reinforcing the variant/trial-counting interplay rather than supplying new format or syntax detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource: 'One-call audit of a strategy's returns', and immediately enumerates the bundled analyses (Reality Check, luck trials, deflated Sharpe, track record, overfitting, OOS decay). This clearly places it as the aggregate counterpart to the granular validate_* siblings, though it never names them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives useful conditional guidance ('with variants, overfitting and out-of-sample decay', 'point returns_file at a CSV instead of pasting'), which tells the agent how inputs change outputs. However, it never states when to choose this one-call audit over the many sibling validators (validate_reality_check, validate_deflated_sharpe, etc.), leaving the primary routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, so the safety profile is covered. The description adds real behavioral context beyond them: validation is conditioned on the count of variants actually run, results are net of costs, and comparison against buy-and-hold, plus an explicit 'not a forecast / not advice' disclaimer. It stops short of describing result structure, but an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and followed by the validation metric list and the disclaimer. Slightly dense in the colon-list clause but no wasted sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and the description covers the pipeline, cost treatment, and the deflated-Sharpe/overfitting/track-record validation angle. What is missing is sibling routing and any note on grid-size limits (that lives in the schema), so it is nearly but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema (grid cap of 200, periods_per_year conventions, family parameter mapping). The description only gestures at the grid and cost concepts and adds no syntax or format detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (backtest) and resource (a rule over a price series via a parameter grid) plus the follow-on validation step, so the agent understands this is a backtest-plus-validation pipeline. It does not, however, name or contrast itself with the many validate_* / audit_backtest siblings, so the agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the description – you call this to run a grid backtest and get validation metrics – but there is no explicit when-to-use vs. the sibling validators (validate_deflated_sharpe, validate_overfitting, audit_backtest), nor any stated prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/openWorldHint/idempotent/destructive, so the bar is lower. The description still adds value: it enumerates the specific rules and models applied (2026 US day-trading rules, square-root impact with crowding) and clarifies the tool performs a pure computation with no broker or account access.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the check and the scope of the model, followed by the isolation constraint. Dense but no filler; the second sentence's boundary statement earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is not needed. For a 15-parameter modeling tool the description conveys the model's components and its read-only, offline nature, though it leaves the roughly one-third of undocumented parameters to the schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, a middle band where the schema documents most inputs on its own. The description gestures at parameter domains (broker order-rate limits, impact, cost capacity) but gives no syntax, units, or defaults beyond what the schema already states, so it neither compensates for the gap nor repeats the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb (check) and resource (a trading plan) with a tightly bounded scope — broker limits, day-trading rules, market impact, and cost capacity — that reads distinctly from the validate_* and backtest_strategy siblings. It does not, however, explicitly name any sibling to differentiate itself from them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Check a trading plan before it trades' gives a clear usage context (pre-trade feasibility screening) and the closing clause states a boundary (never contacts a broker, never sees an account). No explicit when-not-to-use or routing to an alternative sibling is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint=true, idempotent, non-destructive, closed world), so the bar is lower, and the description still adds real context: 'No code is sent' (privacy guarantee) and an explicit coverage caveat about survivorship, vendor revisions, and leaks no cut reaches. That honest scope disclosure is exactly the kind of behavioral detail annotations cannot provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The front-loaded question is good, but the prose is telegraphic and partly unparseable: 'A pass covers only these cuts: the columns are the caller's, and survivorship, vendor revisions or a leak no cut reaches leave no trace' packs three unrelated caveats into one fragment. Information density is high but clarity suffers.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 9 parameters, an enum action, nested columns, and an output schema, the description should at minimum clarify what plan versus compare each return and how the two phases connect. It sketches the workflow and its limits, but omits the distinction between the two actions' outputs and leaves several parameters unexplained, so it is only marginally complete for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, so the baseline is 3. The description touches on cuts, the columns payload, and prefixes implicitly, but adds no meaning for seed, tolerance, timestamps, observations, or columns_file beyond what the schema already documents. It does not compensate for the uncovered third.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening question 'Does a signal look ahead?' and the title state the goal (detecting lookahead bias in a signal), but the description spends most of its text on the mechanics of a two-phase protocol (plan → rerun → compare) rather than crisply stating what the tool returns. An agent can infer the purpose, but the verb+resource framing is buried and the sibling set (placebo_test, validate_overfitting, validate_backtest_length) is never distinguished.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does convey a usage workflow: call plan to get cuts, rerun your code on the first cut rows for each, then compare. That is genuine guidance. However it never says when to prefer this over the numerous validation siblings, nor when a check is unnecessary, so the routing guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are non-readonly, non-destructive, non-idempotent, and closed-world, and the description explains why: 'plan writes placebo files' on your data. It also discloses the key interpretive limit ('does not show the edge survives costs, capacity or the future'), which goes well beyond the annotation set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with a framing question and every clause carries content, but the heavy nested parentheses make it denser and harder to parse than necessary. Still, there is no filler sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter, two-mode tool with an output schema (so return values need no explanation), the description conveys the conceptual workflow and the limits of the result. The remaining gap is that several non-enum parameters are left to the schema, but nothing critical to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 60%, and most parameter meaning (real, placebos default 19 up to 199, columns, data_file) lives in the schema descriptions rather than the prose. The prose adds conceptual context (permuted returns, p-value counting every choice) but says nothing about method, seed, or lower_is_better, so it does not fully compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb and resource: it writes placebo files with returns in random orders, has you run the pipeline on real and placebo data, and compares them for a p-value. This is far more specific than the title. It does not, however, name or contrast the sibling validate_* tools, so sibling differentiation is left to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It implies the workflow (plan writes placebos, then run the pipeline, then compare) and frames the use case as 'Does your pipeline find edges in noise?'. But there is no explicit when-to-use/when-not guidance versus the many sibling validation tools, and the action flow is only implied rather than spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint=false and openWorldHint, so safety is covered. The description adds genuinely useful context — 'No key' (no auth required) and that quotas are returned. But the trailing sentences about the service never seeing data source, costs, survivorship or lookahead describe a validation verdict, not a health check, adding confusion rather than behavioral clarity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded well in the first clause, but two of the three sentences are off-topic content that does not describe this status tool at all. For a zero-parameter health check, those sentences fail to earn their place and actively dilute the definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values (including quotas) need not be explained, and purpose, timing and auth are covered. The definition is adequate but is undermined by the irrelevant series-validation disclaimer, leaving the agent with a slightly muddled picture of what this endpoint actually reports.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4. The only parameter-adjacent statement, 'No key', usefully confirms no credential input is expected, matching the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause states a specific resource and condition — 'Whether the validation API is up, with its quotas' — which cleanly separates it from the validation siblings (audit_backtest, stress_test, etc.). However, the remaining sentences drift into what looks like boilerplate about validating a series, which does not describe a status endpoint and muddies the otherwise clear purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'check after a timeout before resubmitting' gives an explicit triggering condition for calling this tool, and 'No key' removes an auth question. No alternative tools are named, but the when-to-use context is concrete enough that an agent knows exactly when to reach for this over the validation tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world behavior, so the safety profile is covered. The description adds genuine epistemic context beyond that: it reports frequencies (drawdown-limit breaches, negative Sharpe, a fragility share) and explicitly warns that these resampled frequencies are not forecasts of the future. Return-value detail is partly redundant with the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences, front-loaded with the core action and modes. The closing caveat about non-forecasting frequencies is short and earns its place, though the mode list is compressed densely enough to read slightly telegraphically.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 11 optional parameters, full schema coverage, an output schema, and complete annotations, the description only needs to convey purpose, modes, and interpretation caveats, which it does. It could be stronger by stating the required input form (returns or returns_file) or routing among sibling validators, but nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter's meaning and default is already documented in the schema. The description references the drawdown limit and negative-Sharpe condition conceptually but adds no syntax, units, or default information beyond what the schema provides; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (stress) plus resource (a strategy's returns) and enumerates the two modes it runs: bootstrap resampling and named crash/volatility/repeat/stuck-position scenarios. It does not, however, differentiate itself from nearby siblings such as placebo_test or backtest_strategy, which also consume a strategy's return series.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is for robustness/fragility analysis of a return series. There is no explicit when-to-use or when-not-to-use statement, and no sibling tool is named as the alternative for a related question.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds genuinely new context: the output is roughly 100 words plus named fields, and it explicitly scopes the tool by stating it is 'not a forecast or advice' — a useful boundary that annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core purpose before the output inventory and the disclaimer. The first sentence is a fragment ('A long price or return series in about 100 words'), which is slightly compressed but costs nothing in clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, yet the description still names the output fields helpfully. All 9 parameters are documented in-schema and annotations carry the safety profile, leaving only the absence of usage routing as a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% across all 9 parameters, so the schema already documents prices vs returns vs series_file, periods_per_year conventions and benchmark_returns. The description adds no parameter-level detail beyond the generic 'price or return series' phrasing, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (summarize) and resource (a long price or return series), and enumerates the output content (growth, risk, drawdowns, trend, volatility regime, tails, jumps, stale data). It is clearly distinguishable from the validation/backtest siblings, though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The line 'Use instead of reading raw bars' gives a directional when-to-use hint, which is more than nothing. However, there is no explicit when-not guidance, no mention of prerequisites (e.g. minimum series length), and no routing between this and the backtest/validation siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false, destructiveHint=false, openWorldHint=true, so safety is partly covered. The description adds an interpretive caution ('not admission to anything and is not a forecast'), but that sentence concerns deflated Sharpe / overfitting thresholds, which read as properties of sibling tools rather than this one, making the added behavioral signal ambiguous.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is compact and front-loads the core computation, but the final sentence about deflated Sharpe and overfitting thresholds is off-topic for this tool and dilutes focus, and the middle clause is syntactically tangled.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, and the description supplies purpose, planning context, and the sibling handoff. What is missing is any statement of prerequisites or assumptions (e.g., independence of trials) that would let an agent know when the answer is valid.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and all three parameters carry their own descriptions, so the schema already does the heavy lifting. The description's mention of backtest_years ('the most trials those years allow') and target Sharpe adds only marginal framing beyond that baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (minimum backtest length in years) and the quantity it is derived from (N independent trials vs. a target Sharpe), and it hints at the inverse query via backtest_years. However, the phrasing is circular and dense ('the most trials those years allow'), so an agent must already understand the underlying statistics to be sure what the call returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit routing: 'For planning a search; once it has a result, use validate_deflated_sharpe.' That names the alternative and the condition that selects it. It stops short of a full when/when-not matrix (e.g., how it relates to validate_luck_trials or check_feasibility among the many sibling validators).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover safety (readOnlyHint=false, destructiveHint=false, openWorldHint=true), and the description adds genuinely useful behavioral context beyond them: the verdict applies only to the series 'exactly as submitted,' and the service has not seen the data source, costs, survivorship, or lookahead. That is real disclosure of scope and limitations an agent needs to interpret the result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what is computed and followed by scope caveats; little is wasted. The opening sentence is a dense run-on that could have been split, but the information density justifies its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and annotations carry the safety profile. The description adds the key caveat that the verdict applies only to the submitted series and ignores data-source provenance, which is the main interpretive gap an agent would otherwise face.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes further by mapping outputs to inputs: 'this quality and correlation' ties to sleeve_sharpe and average_pairwise_correlation, 'the Sharpe at a sleeve count' to sleeves, and 'the sleeves a target needs' to target. This clarifies the relationship between the optional inputs and which result each drives.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the three quantities computed (Sharpe ceiling from adding sleeves, Sharpe at a given sleeve count, sleeves needed for a target), which is a specific computational purpose rather than a tautology. It also scopes itself against siblings by stating 'it validates no single strategy,' implicitly distinguishing it from the per-strategy validators. The run-on phrasing of the first sentence makes it slightly harder to parse than an ideal verb+resource statement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'For portfolio construction; it validates no single strategy' gives implied usage context, steering the agent toward breadth/portfolio questions rather than single-series validation. However, no alternative tool is named and no explicit when-not condition is given, so the agent must infer the boundary from the sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=false, openWorldHint=true and readOnlyHint=false, so the safety profile is largely covered. The description adds an interpretive caveat ('not admission to anything and is not a forecast') that usefully frames the result as non-decisional, but it says nothing about cost, rate limits, or determinism. With annotations doing the safety work, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core definition before the input modes and sibling routing, and every sentence carries content. It is dense for a single paragraph, but no sentence is wasted; the closing caveat is long yet load-bearing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter statistical validator with 100% schema coverage, an output schema, and annotations present, the description covers input modes, sibling routing, and result interpretation adequately. Return values need not be explained given the output schema, so remaining gaps are minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all eight parameters (including the returns-replaces-four-statistics note) are already documented in the schema. The description only hints at the two input modes ('seven statistics or a return series'), adding marginal value beyond the structured fields. Baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific computation ('probability (0 to 1) that the selected strategy's Sharpe beats the best that luck gives across the variants tried') with its two outputs (probabilistic Sharpe and the luck benchmark). It also routes explicitly to three named siblings, so an agent can distinguish it from validate_overfitting, validate_luck_trials and validate_haircut_sharpe without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives the alternative-input condition ('Send the seven statistics or a return series') and names three alternatives with the condition that selects each: variant-level returns -> validate_overfitting, trial counts -> validate_luck_trials, multiple-testing haircut -> validate_haircut_sharpe. Explicit when-to-use routing, nothing left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false and openWorldHint=true, so the safety profile is covered. The description adds interpretive context — that threshold crossings are not 'admission to anything' and not a forecast — which is genuinely useful caveating. It says nothing about computational behavior such as determinism, limits, or failure modes beyond the schema's bounds.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with what the tool computes, followed by the sibling redirect and the interpretive caveat. Dense with finance jargon, but each clause carries information and nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and the description covers the computation, the sibling alternative, and the interpretation warning. For a six-parameter statistical tool it is adequately complete, though it assumes the reader knows the surrounding multiple-testing framework.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter is already documented, so the baseline is 3. The description's mapping of 'tests' to Bonferroni/independent-test haircuts and 'other_sharpe_ratios_annualized' to Holm/BHY largely restates what the schema descriptions already say, adding little beyond them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific quantity (the Haircut Sharpe for multiple testing, Harvey and Liu 2015) and the exact estimators produced (Bonferroni, independent-test, Holm, BHY), which lets an agent distinguish it from validate_deflated_sharpe. It lacks an explicit verb framing, but the resource and scope are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly routes the agent to the alternative: 'For the probability the Sharpe is real, use validate_deflated_sharpe.' That is a clear use-when-other condition. It stops short of stating the positive condition for when this tool should be chosen (e.g., multiple-testing scenarios with a trial count), but the sibling routing is concrete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=true). The description adds Monte Carlo method context and clarifies that luck is the best of N random tries, plus an interpretation caveat. It does not explain why readOnlyHint is false, but it goes beyond the annotations in useful ways.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three front-loaded sentences: first defines the core output, second gives method and alternative, third adds a caution. No wasted words, though the final sentence is somewhat abstract and could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained. For a complex statistical tool, the description covers purpose, method, alternative, and interpretation caveat; required parameters are left to the schema, which is acceptable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all six parameters are already fully documented. The description mentions 'with a trial count' mapping to effective_independent_trials, but adds no syntax or format details beyond the schema. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific computation: how many skill-less strategies a search would need for its best to reach this Sharpe by luck, and the chance it did with a trial count. It distinguishes itself from validate_deflated_sharpe, so an agent can tell the two apart without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly directs to validate_deflated_sharpe for the probability the Sharpe is real, and warns that a threshold on deflated Sharpe or overfitting probability is not admission or forecast. This gives clear context for when to use this tool versus its sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already declaring destructiveHint=false and openWorldHint=true, the description still adds real context: the required input shape, the meaning and range of the output, and an explicit caveat that crossing any threshold 'is not admission to anything and is not a forecast'. It does not explain determinism/reproducibility of the sampling seed or the combinatorial cost of the run, so it falls short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the metric and method, then the input requirement, then the alternative, then a caution. Every sentence carries weight, though the final disclaimer sentence is somewhat tangential to tool selection.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return structure need not be described, and the description still supplies the output range and interpretation, the input shape, the routing alternative, and the key interpretive caveat. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters (matrix, n_splits, max_combinations, seed) are already documented in the schema. The description only restates the matrix orientation ('periods by variants') and adds nothing about seed, n_splits, or max_combinations, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific metric (probability of backtest overfitting, 0-1) and the exact method (CSCV), plus a one-line definition of what the number means (how often the in-sample best falls below the out-of-sample median). It is immediately distinguishable from the sibling validate_deflated_sharpe.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the data prerequisite ('needs every variant's returns, periods by variants') and names the explicit alternative for the other case ('with summary statistics only, use validate_deflated_sharpe'). This is a clear when-to-use / when-not-to-use routing rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations give a baseline (non-read-only, non-destructive, open-world), and the description adds real context beyond them: local files are never uploaded, CANLI_LOCAL=1 gates record_file/journal_file behavior, and unsigned/unjournalled inputs leave facts and signatures unchecked. It also pre-empts a misread of the verdict ('about the series exactly as submitted'). This is meaningful behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the purpose, then mode behavior, then the scope limitation of the verdict. Every sentence carries information, though the closing sentence about survivorship and lookahead is dense enough to need a second read.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values need no explanation, and the description covers the important gaps for a validation tool: local-file handling, the journal requirement, and the boundary of what the verdict does and does not prove. Complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents the CANLI_LOCAL and journal_file requirements for each parameter, so the description largely restates that gating. Baseline 3 is appropriate; the prose adds only framing, not new syntax or format detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Checks paper-evidence.v0 structure and disclosures') and names the exact format version it operates on, which naturally separates it from the validate_backtest_length / validate_deflated_sharpe family of numeric validators. It stops short of naming a sibling it is not, so it is clear but not maximally differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage via the CANLI_LOCAL=1 mode and the journal/no-journal branch, and notes the consequence of omitting a journal (source facts and signatures unchecked). However it never states the plain when-to-use condition or points to an alternative tool such as verify_receipt, so guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false, destructiveHint=false, openWorldHint=true, so the safety profile is partly covered externally. The description adds interpretive context — that a deflated Sharpe or overfitting probability crossing a threshold is neither an admission nor a forecast — which is genuine value beyond the annotations. It discloses no cost, runtime, or bootstrap-behavior details, so with annotations present a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, purpose front-loaded with the three test names, then the input requirement, then the interpretive caveat. No filler, though the final sentence is semantically loose (mixing deflated Sharpe, overfitting, and 'not a forecast' into one clause) and is more cautionary than actionable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained. The description covers what distinguishes this test family, the critical input requirement, and the interpretive limits of the result — enough for a zero-required-parameter, fully schema-documented tool. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all seven parameters (reps, seed, alpha, matrix, benchmark, matrix_file, block_length) are already documented with defaults and ranges in the schema. The description only reinforces one of them via 'all variants tried' and implies matrix_file can substitute for matrix. Baseline 3 for a fully documented schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names specific tests (Hansen's SPA, White's Reality Check, Romano-Wolf StepM) operating on a clearly stated resource — the returns of every variant a search tried. That is a concrete verb+resource combination an agent can recognize. It does not, however, explicitly route the agent away from the many validate_* siblings (deflated Sharpe, luck_trials, overfitting); it only references a deflated Sharpe as a caveat rather than as an alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is one strong usage requirement — 'Send all variants tried, not only the winners' — which tells the agent the precondition for a valid run, and the scope statement ('on every variant a search tried') implies the data-snooping scenario. But no alternatives are compared or excluded, and the borderline case of many trials vs. the sibling overfitting/deflated-Sharpe tools is left implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=true) and the description does not contradict them. Beyond that, it adds a genuine interpretive caveat that the outputs "are not admission to anything and are not a forecast," which is useful behavioral context about how results should be read.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded and the alternative tool and interpretive caveat follow. It is dense with semicolons and clauses, but each clause carries distinct information and nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is unnecessary, and all parameters are fully documented in the schema. The description supplies purpose, scope, the sibling alternative, and an interpretation caveat, leaving little an agent would need to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all seven parameters with ranges and defaults. The description adds only the conditional meaning of the observations parameter ("with observations, the record's probabilistic Sharpe so far"), which is a marginal gain over the schema's own text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource (minimum track record length in observations and years) and the computation it performs (Sharpe beating a benchmark at a confidence level), and it distinguishes itself by naming the sibling validate_backtest_length for the backtest-sizing case. The core action is expressed as a noun phrase rather than a verb, which slightly softens the clarity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the applicable domain ("For live or paper records") and names the alternative with the condition that selects it ("to size a backtest for its trials, use validate_backtest_length"). This is explicit routing, though it stops short of enumerating when-not conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/destructiveHint, so the safety profile is covered. The description adds substantial non-structured context: verification is offline, the algorithm is Ed25519, the key is bundled/published at a specific well-known URL, and receipts are content-hashed and reproducible from a named open-source core.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and efficiently packed. Slightly redundant, restating the signature/Ed25519/key concept twice and appending a long URL mid-sentence, but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description needn't document return values, and it covers both input paths plus the crypto semantics. The only gap is failure behavior — what a failed or tampered-verification result looks like — which the output schema may or may not cover.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by clarifying the id-vs-receipt relationship (id triggers a fetch, receipt verifies without fetching), which resolves the real ambiguity in a zero-required-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (verify) and resource (receipt), and enumerates exactly what is checked: the Ed25519 signature against a named key, the output hash, and the id. An agent can distinguish this from the validate_* and backtest siblings without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly covers the two input modes — 'Send an id to fetch it first, or the receipt itself' — which is the key usage decision for this tool. It does not, however, say when verification is warranted versus skipping it, nor reference any sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.