Backtest NIFTY option strategies on real 1-minute data, with an honest out-of-sample panel.
Scored across 10 tools
Most tools target distinct resources and actions, but fetch and get_backtest both retrieve by id, and search and describe_coverage both expose coverage information. The descriptions help clarify the differences, though an agent could pause between these pairs.
Most tools follow a verb_noun pattern, but 'fetch' and 'search' are single verbs while 'my_feedback' is a possessive noun phrase. The mixed conventions are still readable and mostly predictable.
Ten tools is a well-scoped size for a backtesting service. Each tool covers a distinct core workflow: running, retrieving, reporting, searching, coverage, strategy listing, methodology, and feedback.
The surface covers the primary lifecycle: run, retrieve, report, search, and give feedback. There is no direct 'list all backtests' or delete operation, but search plus fetch fills that gap adequately for most agent workflows.
Turn a stored backtest into a finished, self-contained Stratify report — one HTML document with the honesty panel, equity and drawdown curves, walk-forward folds, the gross-to-net breakdown, a month grid and the trade table. PUBLISH THE RETURNED HTML VERBATIM AS AN ARTIFACT (Claude), a canvas document (ChatGPT, Gemini), or write it to a .html file (CLI clients). Do not rewrite it, summarise it into your own chart code, or regenerate the figures — the numbers in it came from the backtest, and anything you redraw from a table is a second source that can disagree with the first. It needs no network, no libraries and no build step, and it renders on light and dark. Use it when someone asks for a report, a summary they can keep, something to share, or an artifact.
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | 'artifact' (default) returns the whole document to publish. 'link' returns only the hosted URL — far cheaper in tokens, and the right choice when the user just wants to look at it rather than keep it. 'full' builds the FULL STRATEGY REPORT and returns its link: the strategy's rules in plain English, what it did to ₹10 lakh of capital, every trade plotted on a zoomable NIFTY chart, the evidence panel, and the capital curve. Ask for it whenever someone wants to really understand a strategy rather than glance at it. It is rate limited. | |
| capital | No | format 'full' only. Starting capital in rupees. It sets the report's OPENING view — the reader can change it in the page without a new report. Default 1,000,000. | |
| risk_pct | No | format 'full' only. Size by RISK instead of margin: the percent of capital the trade is allowed to lose in its worst case (2 means 'risk 2% per trade'). Only works where the position has a bounded worst case — a naked short does not, and the call is refused with that reason rather than sized off a guess. Overrides deploy_pct. | |
| deploy_pct | No | format 'full' only. Percent of capital used as margin on any one trade. Default 10. | |
| backtest_id | Yes | From a previous run_backtest. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool returns a complete HTML document that must be published verbatim, and warns against regenerating figures from tables. This adds context beyond the sparse annotations (readOnlyHint=false, destructiveHint=false). It doesn't mention rate limits or side effects, but the schema covers the rate limit for 'full' format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and then provides necessary handling instructions for the returned HTML. Each sentence serves a purpose: defining the output, instructing verbatim publication, warning against redrawing, and stating when to use the tool. It is slightly long but not wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description tells the agent what the output is, how to handle it, and when to use it. It also implicitly covers the main workflow (after run_backtest). Since there is no output schema, the description's explanation of the HTML document's contents fills that gap. Combined with a fully described input schema, this is complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides detailed descriptions for all five parameters, including the format enum's behavior and the purpose of capital, risk_pct, and deploy_pct. Since schema coverage is 100%, the description is not required to add parameter details. It does not, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: turning a stored backtest into a finished, self-contained Stratify report. It names the output type (one HTML document) and lists the included components, making the tool's function unambiguous. It clearly distinguishes from siblings like run_backtest and get_backtest by focusing on report generation from an existing backtest.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use the tool: 'when someone asks for a report, a summary they can keep, something to share, or an artifact.' It also implies it is for already-stored backtests, distinguishing it from run_backtest. However, it does not explicitly state when not to use it or name alternative tools, so it falls short of full guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
What data is available: symbols, date range, resolution, structures, gates, biases, the cost model, and every known gap. Call this before building a spec.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds useful behavioral context by listing what the coverage report contains, including 'every known gap,' which goes beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The first sentence front-loads the answer to 'what data is available' with a precise list, and the second sentence gives a clear call to action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only introspection tool, the description fully covers what the tool returns, why it exists, and when to call it. Nothing essential is missing for an agent to select and invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and 100% schema coverage, so the description has no parameter burden to carry. The baseline for a parameterless tool is 4, and the description adds relevant context about the data domains covered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb-resource pairing ('describe coverage') and enumerates the exact content: symbols, date range, resolution, structures, gates, biases, cost model, and known gaps. This clearly distinguishes it from siblings like run_backtest or fetch, which perform different operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an explicit usage instruction: 'Call this before building a spec.' This is clear context for when to invoke the tool, though it does not explicitly name alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
How a result is produced and how to judge it: entry pricing, settlement, margin, slippage, the honesty rubric, and what each check can and cannot prove. Read this before trusting any backtest, including ours.
| Name | Required | Description | Default |
|---|---|---|---|
| topic | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive behavior, and the description adds useful context about content: it offers a rubric of honesty and clarifies what checks can and cannot prove. It also positions the tool as a prerequisite for interpreting backtests, supplementing the annotation profile without contradicting it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences front-load the purpose, summarize the scope, and end with an actionable directive. There is no filler or redundant restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only explanatory tool with a single optional enum parameter, the description plus the schema gives an agent everything needed to call it correctly: what it covers, why to use it, and valid topic choices. No output schema is needed because the description communicates that this is reference material rather than a data-producing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries some burden for explaining the `topic` parameter, but it does not mention the parameter by name or explain its enum values. It does name several topic areas such as settlement, margin, and slippage, which partially overlap with enum values, providing some guidance but not complete compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly frames the tool as an educational methodology reference, specifying the resource ('how a result is produced and how to judge it') and concrete topics such as entry pricing, settlement, margin, slippage, and the honesty rubric. It also distinguishes itself from siblings, none of which explain methodology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The final sentence gives an explicit trigger: 'Read this before trusting any backtest, including ours.' This tells an agent when to call the tool, though it does not enumerate exclusions or name a specific alternative tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch a document or backtest result by id, as returned by search.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint=true and destructiveHint=false, so the description does not need to repeat those. It adds useful context that the id must come from search results, but does not disclose behavior on invalid ids or return details. Given the annotations cover the safety profile, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence with a clear verb and object, front-loading the action and resource. No filler or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple fetch-by-id tool with annotations, the description covers purpose and id provenance. However, the existence of a sibling named get_backtest makes the lack of disambiguation a notable gap. The description would be more complete if it mentioned when to use get_backtest instead, or explained the difference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It meaningfully specifies that the 'id' is the identifier returned by search, clarifying the parameter's origin and expected type. While it does not describe format or validation, it provides essential semantic context beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action (fetch) and resource (document or backtest result), and adds provenance ('as returned by search') to clarify the id source. It is not a tautology and distinguishes from typical search tools, but it does not explicitly differentiate itself from the sibling get_backtest, which likely fetches backtest results specifically.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'as returned by search' implies a workflow but does not explicitly state when to use this tool versus alternatives like get_backtest or search. No exclusions or alternative recommendations are provided, leaving the choice between fetch and get_backtest ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve a previous backtest result by its id — honesty panel, equity curve and per-trade detail, exactly as first computed.
| Name | Required | Description | Default |
|---|---|---|---|
| detail | No | ||
| backtest_id | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds meaningful behavioral context by stating 'exactly as first computed,' indicating the returned result is the original stored output rather than a fresh or modified computation. It also names the major return components, which is useful beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly worded sentence that front-loads the action and object, then adds useful detail in the em-dash clause. There is no fluff or repetition of already-available schema information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only retrieval tool with two parameters output schema is absent, so the description must cover return expectations—which it does at a high level. However, the optional detail parameter is completely unexplained, leaving a gap in how the agent can control the amount of returned information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clarifies that backtest_id identifies the stored result, but it says nothing about the optional detail parameter or what 'summary', 'standard', or 'full' return. This leaves an entire documented parameter without operational meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Retrieve a previous backtest result'), the resource ('by its id'), and the expected content ('honesty panel, equity curve and per-trade detail'). The phrase 'exactly as first computed' distinguishes it from recomputing a backtest, which differentiates it from siblings like run_backtest.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the appropriate context: use this when you need an already-computed backtest result rather than creating a new one. It does not explicitly mention alternatives or when-not-to-use conditions, but the 'previous backtest result' framing gives clear contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Strategies from THIS account's history that held up under out-of-sample and walk-forward checks, not merely ones that made money. Ranked by worst walk-forward fold — consistency, not size. Call it to answer 'what has worked for me so far?' without re-running anything.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| order | No | 'consistency' (default) sorts by worst walk-forward fold, then median fold. 'pnl' sorts by total P&L and is the ranking most likely to put an overfit at the top. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true and destructiveHint=false. The description adds meaningful context about the selection criteria (only validated strategies), the ranking logic (worst walk-forward fold, not P&L size), and the caveat that 'pnl' ranking may surface overfits. This goes beyond the schema and annotations, although it doesn't describe return formats or pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The first sentence defines scope and ranking, the second states the use case. It is front-loaded and every word contributes to understanding the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with no required parameters and no output schema, the description covers the essentials: what is listed, how it is ranked, and when to call it. The annotations handle safety, and the schema covers 'order'. The only minor gap is that the return structure is not described, but that is not critical for a filtered-list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description covers the 'order' parameter in detail, including the overfit warning for 'pnl'. The tool description reinforces the consistency ranking but does not explain the 'limit' parameter. With 50% schema coverage, the description adds only marginal parameter-level value beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states exactly what the tool does: it lists strategies from the account's history that passed out-of-sample and walk-forward checks, ranked by worst walk-forward fold. It distinguishes itself from siblings like run_backtest and get_backtest by emphasizing the validation filter and the 'without re-running anything' use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use the tool: 'to answer what has worked for me so far?' and 'without re-running anything.' However, it does not explicitly name alternative tools or state when not to use it, so it stops short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reports this account has filed, and where each one stands. Use it to answer 'did that bug I reported ever get fixed?'.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds that it returns status ('where each one stands'), which informs the agent that it provides tracking info, not just a list. However, it doesn't describe return format or ordering, but that's not expected for a simple read tool with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. The first sentence states the function and output, the second gives an example query that grounds it. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, clear annotations, and a simple read purpose, the description is complete. It could mention that it returns feedback statuses for the current account only, but that's already implied by 'this account has filed.' The sibling context (submit_feedback) helps disambiguation. A 4 is appropriate for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100%, so there are no parameter semantics to add. The description compensates by explaining what the tool returns and its purpose. No additional parameter guidance is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Reports this account has filed, and where each one stands.' This distinguishes it from submit_feedback, which creates feedback. The 'reports this account has filed' is clear, though it could specify it lists or retrieves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a concrete use case ('did that bug I reported ever get fixed?') that implies when to use it. It does not explicitly name alternatives or exclusions, but the sibling context makes it clear this is for viewing personal submissions, distinct from submit_feedback. The guidance is useful but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Backtest an Indian index option strategy on real 1-minute NIFTY options data. Returns P&L after real charges and slippage, return-on-margin, and an honesty panel: out-of-sample split, walk-forward folds, bootstrap interval, and a deflated Sharpe that accounts for how many variants you have already tried. Refuses windows too narrow to be meaningful, and reports no ratios below 30 trades. Two spec forms: a PRESET (structure + params) for the common shapes, or an OPEN STRATEGY (legs + rules) for anything else — any number of legs at any strikes on any expiry, strikes chosen by percent, points, premium or delta, entry at any minute, and rules that CHANGE the position while it is live (roll a tested leg, close one side, add a hedge, trail a stop) plus book-level rules like standing down after three losers. Nothing here is restricted by tier; a paid tier only widens the date window.
| Name | Required | Description | Default |
|---|---|---|---|
| lots | No | ||
| spec | Yes | Either a preset spec (structure + params) or an open strategy (legs + rules). Use the open form for anything the presets cannot say. | |
| detail | No | How much per-trade data to return. 'standard' (default) is the equity curve, breakdowns and the first 25 trades with their leg prices. 'full' returns up to 300 trades — ask for it when the caller wants to audit or chart every trade. 'summary' returns aggregates only, and is the cheapest to read. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/destructive hints, the description discloses meaningful behavioral details: real charges and slippage, the honesty panel components, refusal thresholds, and that the OPEN STRATEGY form can simulate changing positions while a trade is live. It also addresses tiering ambiguity by stating that paid tiers only widen the date window. This substantially exceeds what the annotations alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded, opening with what the tool does and what it returns. The long list of capabilities is mostly justified by the tool's complexity, and each clause adds useful information. Still, the paragraph is quite dense and could be trimmed slightly without losing essential meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema present, the description wisely covers the return value: P&L after charges and slippage, return-on-margin, and the honesty panel. It also clarifies spec forms, refusal conditions, and tier behavior. It does not explain how `detail` or `lots` affect calls, but the schema covers those, and it doesn't connect to `get_backtest` for retrieving past results, so a small completeness gap remains.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, so the description shares the burden of explaining parameters. It adds high-level meaning for `spec` by explaining the preset vs. open-strategy split, strike selection modes (percent, points, premium, delta), and live-changing rules. However, it never names top-level parameters like `lots` or `detail`, leaving much of the detailed param semantics to the schema. This is adequate but not fully compensating.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Backtest an Indian index option strategy on real 1-minute NIFTY options data.' It also enumerates the return contents (P&L, return-on-margin, honesty panel), so the tool's purpose is unmistakable. However, it never explicitly contrasts this tool with siblings like get_backtest or build_report, so it stops short of full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear in-tool guidance: use a PRESET for common shapes or an OPEN STRATEGY 'for anything else,' including concrete examples such as rolling a leg or adding a hedge. It also sets expectations about when it refuses to run (narrow windows, fewer than 30 trades) and how tiering works. It does not name sibling-tool alternatives explicitly, so it is not a perfect 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Search what this service covers — symbols, dates, structures, signals, methodology. Returns ids usable with fetch.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds the useful fact that the tool returns ids usable with fetch, but it does not disclose result limits, pagination, or any errors/edge cases. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences, front-loaded with the main purpose, and every phrase adds information: the coverage scope and the downstream use of the returned ids. There is no repetition of the tool name or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter, read-only search tool, the description covers purpose, scope, and output, which is enough to attempt a call. However, without an output schema it only vaguely says 'returns ids,' leaving the result shape and query syntax underspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required string parameter, 'query', with no description (0% schema description coverage). The description mentions the searchable categories but never explains how to format query, whether it supports partial matches, wildcards, or exact identifiers, so the agent must guess at query semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action ('Search') and a clear resource ('what this service covers'), and enumerates the dimensions it covers: symbols, dates, structures, signals, methodology. It also states the output (ids usable with fetch), but does not explicitly differentiate it from sibling tools like describe_coverage or explain_methodology.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for discovering what data the service offers and for obtaining ids to use with fetch, so the intended use is inferable. However, it gives no explicit when-to-use vs alternatives, no exclusions, and does not mention sibling tools such as describe_coverage or list_strategies.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report a bug, request a feature, flag a data gap, or say what worked. Use this whenever the user expresses a problem with this service or wishes it did something it does not — do not just apologise to them, file it. If a backtest was involved, pass its backtest_id: that attaches the exact spec and the recent call trail so the issue can be reproduced without a reply. Tell the user you filed it and give them the returned id.
| Name | Required | Description | Default |
|---|---|---|---|
| body | Yes | What was expected, what happened, and any spec involved. Write it from the user's report, not from your own summary of it. | |
| title | Yes | One line naming the problem or request. | |
| category | No | Omit it and it will be inferred from the text. | |
| severity | No | ||
| backtest_id | No | The result this is about, if any. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false and destructiveHint=false. The description adds useful behavioral context: it files a report (creating a record), instructs to 'Tell the user you filed it and give them the returned id,' and explains that backtest_id attaches the spec and call trail for reproducibility. No contradictions; it supplements the annotations well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than typical but every sentence earns its place: purpose, usage trigger, backtest handling, and user communication. It is front-loaded with the core purpose and flows logically. It is appropriately sized for the complexity of a feedback tool with multiple categories and a conditional field.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains the expected return behavior ('give them the returned id'). It covers the main operational details: what triggers use, how to handle backtest contexts, and what to communicate to the user. Nothing an agent needs to correctly invoke and follow up is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 80%, giving a baseline of 3. The description adds meaningful guidance beyond the schema: body should be 'from the user's report, not from your own summary,' and backtest_id is explained as attaching 'the exact spec and the recent call trail.' It also notes category can be inferred when omitted, which is not in the schema. These enrich parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Report a bug, request a feature, flag a data gap, or say what worked.' It clearly states what the tool does and enumerates the covered categories, distinguishing it from siblings like my_feedback (which retrieves feedback) and other unrelated tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'Use this whenever the user expresses a problem with this service or wishes it did something it does not' and even instructs 'do not just apologise to them, file it.' It also provides a conditional rule for including backtest_id when a backtest is involved. This fully clarifies when to invoke the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
build_reportdescribe_coverageexplain_methodologyfetchget_backtestlist_strategiesmy_feedbackrun_backtestsearchsubmit_feedbackHonest A-F grades for trading strategies, backtested on real out-of-sample data. No hype.
Backtest plain-English trading strategies on real market data: graded results, honesty flags.
Backtest strategies and analyze portfolios on any ticker: CAGR, drawdown, Sharpe, from real data.
Crypto backtesting tools: real backtests with robustness verdicts, daily signals and market data.
Add one secure layer between your agents and this server.
