Server Quality Checklist

Tool Scores

• strategyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, and the description adds useful input-resolution details (URL slug vs `algorithm_uid`) plus precise return semantics such as `cagr` in local currency and `cagr_usd` as the canonical USD metric. However, it ends with an incomplete 'Raises a ...' and does not disclose error conditions, auth requirements, or rate limits, leaving a clear behavioral gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The opening sentence is strong and the usage hint is useful, but the description repeats the entire identifier/returns/usage block verbatim, which is redundant and confusing. The incomplete 'Raises a ...' clause also breaks the structure, so the description is noticeably longer than it needs to be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter, read-only tool with an output schema, the description covers identifier formats, exact returned fields, and the workflow trigger, which is nearly complete. The main missing piece is the truncated error clause, and the duplication adds noise, so it does not earn a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry all parameter meaning. It does this thoroughly: `identifier` is defined as either a URL slug (`ma-crossover`) or an `algorithm_uid` ULID (`01J7...`), with examples and format details. No parameter semantics are left to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb+resource: 'One strategy's full dossier, including its backtest summary.' It also specifies the retrieval key (`identifier`) and enumerates the returned metadata, making it easy to tell this detail lookup apart from the `leaderboard` listing. It does not explicitly differentiate from every sibling (`research`, `instrument`), but the title and single-parameter contract make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit workflow guidance: 'Use this after `leaderboard` once a candidate is chosen, or when the user names a strategy directly.' That clearly situates the tool in a sequence and distinguishes it from a browsing/listing action. It stops short of a full 5 because it never states when not to use it or mentions other sibling tools as alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• researchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description discloses substantial behavior: it returns two ranked passage lists, explains the paid-tier knowledge corpus behavior, describes locked responses with an unlock URL, and tells the agent to mention the paid sign-in. This goes well beyond what annotations or the schema provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with purpose, usage, then return details and args. It is longer than minimal but every section earns its place; the only minor deduction is some redundancy between the opening summary and the detailed return-list explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter read-only search tool with an output schema, the description is complete: it covers query intent, locale behavior, result shapes, citation URL construction, paid-tier locking behavior, and how to handle locked responses. No critical operational detail is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0%, the description fully compensates by documenting each parameter: query semantics and length, locale allowed values, and limit meaning. This adds practical meaning beyond the raw schema constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Search the research'), clarifies the two-corpus behavior, and provides concrete example topics. This distinguishes it clearly from sibling tools like strategy or leaderboard, which are not knowledge-search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Use when the user asks about a topic the research likely covers') and gives example topics. It does not explicitly state when not to use it or name alternatives, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• leaderboardA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only say `readOnlyHint: true` and `openWorldHint: false`. The description adds meaningful behavioral context beyond that: no risk filtering is possible, `unit_risk_band` cannot be filtered or sorted, `symbol` matching is case-insensitive with ticker normalization, and sort options have specific well-defined meanings. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well structured and front-loaded with the core purpose and usage examples. Every paragraph earns its place: symbol behavior, risk-filter caveat, sorting semantics, and return fields. A few clauses are somewhat redundant with the schema, but overall it is efficiently organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the key operational details an agent needs: when to use the tool, how sorting works, how `symbol` affects `best_run`, what fields are returned, and an important limitation (no risk filter). An output schema exists, so the description need not restate return structures. This is complete enough for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate and does for the two most nuanced parameters: `sort` is fully explained with its enum meanings, and `symbol` gets detailed normalization semantics. However, `family`, `status`, `limit`, and `offset` receive little or no explicit textual explanation, though their schema titles and enum values make them partially self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'published, backtested strategies, ranked' on a leaderboard. It clearly differentiates from the sibling `strategy` tool by noting that a single strategy's full dossier belongs there, so an agent can distinguish the two without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly gives example user intents ('what are the best strategies?', 'what works best on AAPL?') and explains when `symbol` is appropriate. It also states when not to use this tool for risk filtering and directs users to the `strategy` tool for a full strategy dossier, providing clear when/alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• instrumentA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds useful behavior beyond that: the catalog/dossier mode switch, case-insensitive symbol matching, full-slug disambiguation, and `stats` being null for newly added instruments. No contradiction exists between the description and annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core distinction—catalog vs. dossier—then organized by mode, with fields, examples, and edge cases in a logical order. Every section earns its place, and the closing `leaderboard` routing is useful without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only catalog/dossier tool with an output schema present, the description covers both operation modes, filter dimensions, symbol resolution edge cases, null stats behavior, and sibling routing. An agent has everything it needs to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the burden of explaining parameters. It thoroughly explains `symbol` (case-insensitive, slug pinning), `asset_class` (including enum values), and `status` as a filter. However, it does not explicitly describe `limit` or `offset`, though their names and schema defaults make them fairly obvious.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: it returns either the instrument catalog (when `symbol` is absent) or one instrument's dossier (when `symbol` is provided). It distinguishes itself from siblings by naming the `leaderboard` alternative and describing the exact fields returned in each mode.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit 'Use for...' examples for both modes, such as 'list your crypto instruments' and 'what do you know about AAPL?'. It also explicitly routes strategy backtest queries to the `leaderboard` sibling with the same symbol, providing clear when-to-use vs. when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.