Server Quality Checklist

Tool Scores

• xql_explainA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the output type (execution plan and explanation) but does not explicitly state that the query is not executed, nor describe any side effects, permissions, or error behavior. Given the tool's simplicity, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 15-word sentence that leads with the key actions and includes the output in a compact, easily parsed format. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter, no output schema, and no annotations, the description provides the essential information: what it does and what it returns. It lacks guidance on when to choose this over related tools, but the core functionality is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description 'Raw XQL query text' fully covers the single parameter, so baseline is 3. The tool description reinforces that the input is an XQL query but adds no additional syntactic or semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs 'Parse and plan' and identifies the resource as 'an XQL query'. It clearly differentiates from siblings (xql_validate, xql_run) by stating it returns an execution plan and explanation, which is neither validation nor execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit usage guidance or alternatives are mentioned. The description's implied use case is understanding query execution, but it does not contrast with xql_validate or xql_run or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• xql_validateA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the transparency burden. It discloses the key non-execution behavior, which is valuable safety context. However, it does not describe what happens on invalid input (e.g., error handling) or any side effects beyond planning.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action and resource. Every word contributes value, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description covers purpose, input, and key behavioral constraint. However, it omits what the tool returns or how success/failure is indicated, which is a notable gap given there is no output schema to fill this in.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'query' parameter. The tool description adds no additional semantic meaning beyond what the schema already provides, matching the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates XQL query text by parsing and planning it. The phrase 'without executing it against live data' explicitly differentiates it from sibling tool xql_run, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for checking query validity before execution. The qualifier 'without executing it against live data' provides clear context for when to use it, though it does not explicitly name alternative tools or exclusion scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• xql_runA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It mentions 'live-data round trip' and 'executing against live market data', implying cost/latency, but does not disclose whether the operation is read-only, potential error modes, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose and followed by a concise usage instruction. Every sentence contributes value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema, the description adequately explains the action, return type (matching instruments), and the recommended validation workflow. It lacks details on error handling or response format, but these are not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the query parameter as 'Raw XQL query text' (100% coverage). The description adds context about executing against live data, but does not significantly augment the schema's parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('execute') and resource ('XQL query against live market data'), and distinguishes it from siblings by mentioning xql_validate as a pre-step. It accurately describes the return of matching instruments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to call xql_validate first to catch syntax errors before using this tool, providing clear when-to-use guidance and an alternative. This differentiates it from xql_validate and implies the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• xql_nl_referenceA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It states the operation is a fetch (read-only) and describes the content and purpose. It doesn't mention potential absence of side effects or error behavior, but the simple retrieval nature is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with no fluff. The first sentence packs concrete examples of what the reference contains; the second provides direct usage guidance. Every phrase earns its place relative to the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema, no annotations), the description is complete: it states the return type (reference document), key contents, and when to call it. It also references sibling tools appropriately, fully covering the context needed for correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the baseline for 0-param tools is 4. The description doesn't need to explain parameters; the schema confirms no inputs. The description adds semantic value by explaining the tool's role, which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the XQL natural-language reference document and specifies its contents (mapping retail phrasing to syntax, hard limits). It distinguishes from siblings (xql_validate/xql_explain/xql_run) by positioning itself as the reference lookup rather than an execution or validation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: call before writing a query, or after sibling tools return isError: true. This directly addresses when to use this tool versus the alternatives, giving clear context and an actionable trigger.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.