Server Quality Checklist

Tool Scores

• compare_etf_snapshotsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It does state one important constraint ('never emit a buy recommendation'), but it fails to mention other traits such as whether the tool makes external calls, requires authentication, handles invalid JSON, or has side effects. This is minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that front-loads the purpose and includes a useful guardrail. Every word earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has two JSON inputs and an output schema, but the description is too terse to fully guide an agent. It omits input format expectations, potential errors, and when to prefer this tool over siblings. The description relies heavily on the tool name and schema to convey completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain what left_json and right_json should contain or how they map to 'comparable ETF facts'. The parameter names imply two ETF snapshots, but the description adds no additional detail about expected format or structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and clearly identifies the resource ('comparable ETF facts and warnings'). The tool name 'compare_etf_snapshots' reinforces the comparison function, distinguishing it from siblings like validate_etf_snapshot and audit_backtest_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus the siblings. The description does not mention scenarios, alternatives, or exclusions. The only extra instruction ('never emit a buy recommendation') is a behavioral constraint, not usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_backtest_reportB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It states the tool audits a report but does not describe any side effects, permissions needed, output format, or behavior when gates fail. The presence of an output schema partially mitigates return-value ambiguity, but other behavioral aspects remain undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the essential purpose without any redundant words or filler. It is appropriately concise for a tool with a single parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has low complexity (one parameter) and an output schema, so the description's brevity is acceptable. However, it lacks usage guidance and behavioral detail, leaving some context gaps. It is a minimal viable description but falls short of being comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines only 'report_json' as a string, but the description does not explain how this parameter relates to the BacktestReport being audited. With 0% schema description coverage, the description should compensate but fails to add any meaning about the parameter's content, format, or usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'audit' and the resource 'BacktestReport', and specifies the scope of the audit ('timing, cost, split, and evidence gates'). This distinguishes it from sibling tools like validate_etf_snapshot or compare_etf_snapshots, which target different resources or actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus the sibling tools, nor does it mention exclusions or prerequisites. It only states what the tool does, leaving the agent to infer usage without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_etf_snapshotA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses a key behavioral trait: validation does not fill missing values. However, it lacks details on error handling, return format (though output schema exists), or whether the operation is strictly read-only. Some value is added, but significant gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the action verb. Every word adds value, and there is no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter, an output schema (which covers return values), and clear sibling differentiation, the description is largely complete. It conveys the core function and an important behavioral nuance. It falls short of full completeness only by omitting explicit guidance on when to choose this over siblings, but given the simplicity, this is not a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema exposes one parameter, snapshot_json (string), with 0% coverage from the description. The description adds meaning by indicating the parameter represents an ETFAnalysisSnapshot, providing context beyond the bare schema. However, it does not clarify whether the expected input is a JSON string or its exact format, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Validate') and resource ('ETFAnalysisSnapshot'), scoped to 'one' snapshot. It also includes a distinctive qualifier ('without filling missing values') that helps differentiate it from sibling tools like compare_etf_snapshots.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for validating a single snapshot and explicitly notes a behavioral constraint (no missing value imputation). However, it does not explicitly state when to use this tool over alternatives, nor does it mention any exclusions or conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finance_kit_doctorA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It states the tool returns capabilities and proves the absence of order tools, but it does not disclose output format, side effects, prerequisites, or any potential concerns. For a read-only introspection tool, this is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy, front-loaded with the verb 'Return,' and communicates the core action and purpose efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description covers the essential purpose. However, it does not elaborate on how the proof is presented or whether it returns a full capability list or only a binary confirmation, leaving slight ambiguity about the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline score is 4. The description adds no parameter details, but the empty schema confirms there are none, so this is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' with resource 'capabilities' and explicitly states the unique goal of 'prove that this server exposes no order tools,' which clearly distinguishes it from the validation/audit sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case (checking whether order tools exist) but does not explicitly state when to use this tool versus alternatives or provide exclusions. No alternative tools are mentioned, leaving usage context implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.