Server Quality Checklist

Tool Scores

• run_tournamentA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only describes basic mechanics and output, but omits important behavioral traits like side effects, permissions, computational cost, or state changes. This is insufficient for a tool that likely executes user-provided code.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no redundancy: first states action and type, second clarifies round-robin format, third describes output. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately mentions return structure (leaderboard, wins/losses, match results). However, it lacks context on minimum number of strategies, handling of both strategy inputs simultaneously, and performance implications. Coverage is basic but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with descriptions for all parameters. The description adds no extra semantic value beyond the schema, such as interaction between parameters or constraints. Baseline score of 3 is appropriate since schema already documents parameters well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool runs a round-robin tournament, explains it's between multiple strategies, and specifies the output (leaderboard with wins/losses and match results). Distinguishes from siblings like run_battle (single match) and list_strategies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for round-robin tournaments but does not explicitly state when to use vs alternatives (e.g., run_battle for single matches, or other tournament formats). No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_strategy_codeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. States it's for getting source code (read operation) but doesn't disclose any behavioral traits like side effects or authentication. Adequate for a simple read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Directly states purpose and usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one simple parameter and no output schema, the description is mostly complete. However, it does not describe the return format (e.g., source code as a string), which could aid usability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with examples for the 'id' parameter. Description does not add additional meaning beyond what the schema already provides, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Get the source code of a specific strategy by its ID', specifying verb and resource. Distinguishes from siblings like list_strategies (which lists strategies) and run_battle (which runs battles).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'useful for analyzing how a strategy works', indicating when to use. However, no explicit exclusions or mentions of alternatives like list_strategies for finding IDs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_strategiesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is adequate for a simple read-only listing operation, but with no annotations, it could provide more detail about side effects (none), rate limits, or response size. The lack of any behavioral caveats beyond listing is acceptable but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that efficiently conveys purpose and return data. No extraneous information. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, the description covers the main purpose and return fields. It could mention that it returns a list or array, but the verb 'returns' implies that. Completeness is good but not flawless.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the schema coverage is effectively 100%. The description adds no additional parameter information, which is appropriate given zero parameters. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'list', resource 'battle strategies', and explicitly mentions returned fields (IDs, names, fighting styles, sources). It clearly distinguishes from sibling tools like get_strategy_code or run_battle.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use this tool (to list available strategies), but does not explicitly state when not to use it or suggest alternatives. However, context from sibling tools makes usage boundaries clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_battleA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It explains the action (run battle), optional turn log, and return values. It does not mention side effects, auth requirements, or error handling, but the behavior is clearly a simulation with no persistent state changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose and key options. It is compact, readable, and contains no unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains the return values (winner, HP, turns) and optional includeTurnLog. It does not cover error conditions or rate limits, but for a simulation tool with no required params, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters well. The description adds overall context (returns winner, HP, turns) and the includeTurnLog option, but does not add significant meaning beyond the schema descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a 1v1 battle between strategies, specifying the options (by ID or custom code) and the return values (winner, HP, turns). This distinguishes it from sibling tools like run_tournament, which runs multiple battles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions using list_strategies to get IDs and provides an alternative of custom code. It implicitly suggests when to use this tool (single battle) vs run_tournament (multiple battles). However, it does not explicitly state when not to use it or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_game_rulesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description is the sole source. States the tool returns data and lists example contents, but does not explicitly state it's read-only or side-effect free. Adequate for a simple getter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with key information, no wasted words. Perfect conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless, no-output-schema tool, the description fully covers what the tool does and why it's useful. Complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters; schema coverage is 100%. Description does not need to add parameter info. Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Get the game rules and constants' with examples like damage values, ranges, stamina costs. Differentiates from siblings such as get_strategy_code and list_strategies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Useful for understanding the game mechanics when designing strategies,' providing clear context. Does not name alternatives but the sibling list makes differentiation implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.