Server Quality Checklist

Tool Scores

• list_editionsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states what the tool does and the return type, missing critical details like whether it's read-only, requires authentication, has rate limits, or pagination behavior. For a tool with no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the main purpose stated first and a brief note on returns. It avoids unnecessary elaboration, though the 'Returns:' section could be slightly more informative. Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema exists), the description is adequate but lacks depth. It covers the basic purpose and return type, but without annotations, it misses behavioral context. The output schema helps, but the description could better explain what 'Information about all available editions' entails, such as format or scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there's no need for parameter details in the description. The description correctly avoids unnecessary parameter information, earning a high score for not adding redundant content beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all available editions and their descriptions.' It specifies the verb ('List') and resource ('editions'), making the action explicit. However, it doesn't differentiate from sibling tools like 'get_strategy' or 'search_strategies', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or contexts where this tool is preferred, such as for browsing all editions versus searching or retrieving specific ones. This lack of comparative usage information limits its helpfulness.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_strategiesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that the search is 'case-insensitive', which is useful behavioral context. However, it doesn't disclose other important traits like whether this is a read-only operation, potential rate limits, authentication needs, or pagination behavior for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The 'Args' and 'Returns' sections are structured clearly, though the 'Returns' section could be slightly more detailed. Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, search functionality), the description is fairly complete. It explains parameters and return values, and an output schema exists, so detailed return explanations aren't needed. However, without annotations, it could benefit from more behavioral context like safety or performance notes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. The 'Args' section explains that 'query' is for 'keywords to search for (case-insensitive)' and 'edition' is 'optional' to 'limit search to', clarifying semantics that aren't in the schema. With 2 parameters and good coverage in the description, this compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for strategies containing the specified keywords.' It specifies the verb ('search') and resource ('strategies'), but doesn't explicitly differentiate from sibling tools like 'get_strategy' or 'list_editions', which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the 'Args' section, which explains when to use the optional 'edition' parameter, but it doesn't provide explicit guidance on when to choose this tool over alternatives like 'get_strategy' or 'list_editions'. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_strategyA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool returns a random strategy, which is useful, but doesn't describe other behavioral traits such as whether it's idempotent, has rate limits, requires authentication, or what happens on errors. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose in the first sentence and parameter details in a structured format. Every sentence earns its place by providing essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter) and the presence of an output schema (which handles return values), the description is mostly complete. It covers purpose, parameter details, and return format at a high level. However, it could be more complete by addressing behavioral aspects like randomness or error handling, given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It provides the parameter 'edition' with a clear list of allowed values (edition-1 through edition-4, condensed, programmers, do-it) and specifies the default as edition-2, adding meaningful semantics beyond the bare schema. This adequately covers the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get a random oblique strategy') and resource ('from the specified edition'), distinguishing it from sibling tools like list_editions and search_strategies. It precisely defines what the tool does without being vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (to get a random strategy from a specific edition), but it doesn't explicitly state when not to use it or mention alternatives like search_strategies for non-random searches. The context is clear but lacks explicit exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.