Server Quality Checklist

Tool Scores

• predict_deadlock_draftA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains that the prediction is 'calibrated' (not a guess) and instructs to 'Report verbatim' for the returned win-rate percentages. It also notes internal hero name normalization. This provides good behavioral context, though rate limits or error cases are not discussed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a lead sentence stating the core function, a supporting sentence about calibration and model source, then explicit parameter documentation, and a final note on return values. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 heroes per team, calibrated model), the description covers purpose, input requirements, and output nature. It lacks details on error handling or invalid inputs, but for a straightforward prediction tool, it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so parameters lack descriptions in the schema. The description explicitly documents both parameters (team0_heroes, team1_heroes) as 'Team 0's 6 heroes (names/aliases)' and similarly for team1, adding meaning beyond the schema's array type. This compensates well for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it predicts 'calibrated win probability for a Deadlock 6v6 draft', using specific verb 'predict' and resource 'win probability'. It distinguishes itself from sibling tools (e.g., predict_dota_winrate) by specifying Deadlock context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Provide 6 heroes per team' and 'Backed by batru.gg's Deadlock production model', implying use for Deadlock drafts. However, it does not explicitly state when not to use or compare with alternatives, leaving usage context implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommend_dota_pickA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses calibrated win rates, internal normalization, and return format. It does not mention any destructive actions or error conditions, but is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with a lead sentence, context, and clear Args section. It is concise and front-loaded with main purpose. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no output schema, and sibling tools, the description is fairly complete. It explains what, how, and output. Lacks explicit when-not-to-use, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining each parameter: my_heroes (names/aliases, 0-4), enemy_heroes (0-5), my_side with default. It adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool recommends the top 3 Dota 2 heroes to pick next, with calibrated win rates. It uses specific verb and resource, and differentiates from siblings like predict_dota_winrate and get_dota_counters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on parameters and return format, but does not explicitly state when to use this tool vs alternatives or when not to use it. The Args section helps, but no direct exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_heroA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that backend silently drops unrecognized names and that this tool prevents that. Describes return fields (id, displayName, shortName, game) and error with did_you_mean. No contradictions; no annotations provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet complete: one-sentence summary, usage guideline, technical details, and structured Args/Returns. Every sentence adds value. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description explains return structure and error case. Covers game parameter values, silent drop behavior, and use case. Fully addresses tool complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, so description carries full burden. It describes 'query' as 'A hero name, alias, or short name' and 'game' as 'dota2 (default) or deadlock'. Adds concrete meaning beyond raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it normalizes a hero name/alias/shortName to canonical identity. Gives examples ('am', 'anti mage'). Distinguishes from sibling prediction tools by positioning it as a preprocessing step before calling them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use this tool to turn messy input into exact key expected by prediction tools. Emphasizes always normalizing first because backend silently drops unrecognized names. Does not mention when not to use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dota_countersA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully covers behavior: it uses empirical data from real games, normalizes hero names, and returns win rates with sample sizes. It is honest about being a read-only lookup and explains the meaning of winrate_pct.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a summary line, then explanation of best/worst, then Args and Returns sections. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description fully defines the return object shape and field meanings, including how to interpret winrate_pct. All parameters are covered, and the tool's context is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no descriptions in JSON Schema), but the description compensates by explaining both parameters: hero accepts name/alias/shortName, and limit defaults to 12. It adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the strongest matchups (counters) for a Dota 2 hero from real games.' It uses a specific verb and resource, and the distinction from sibling tools like 'lookup_hero' or 'predict_dota_winrate' is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool returns and advises to 'Report numbers verbatim,' implying usage context. However, it does not explicitly state when not to use this tool or compare it with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• predict_dota_winrateA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses calibration (model is calibrated so reported numbers are empirical), that hero names are normalised, partial drafts supported, and that returns should be reported verbatim. This is thorough behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a clear opening sentence, then a paragraph of important context, and a bullet-like Args section. Every sentence adds value; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no output schema, and no annotations, the description fully covers inputs, behavior, calibration, and return instructions. It answers likely agent questions about partial drafts, empty drafts, and output handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains each parameter in detail: my_heroes and enemy_heroes are arrays of hero names/aliases (0-5), my_side defaults to 'radiant' and specifies which side is my_heroes. This adds full meaning beyond the schema's minimal title and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Predict the CALIBRATED win rate for a Dota 2 draft.' It specifies the exact action (predict), resource (Dota 2 draft), and adds nuance (calibrated). Siblings like lookup_hero or recommend_dota_pick are distinct, making this tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (e.g., for draft win rate prediction), notes that partial drafts are fine and empty draft returns 50/50. However, it does not explicitly contrast with siblings or state when not to use it. The sibling names provide context but the description itself could be more explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.