Server Quality Checklist

Tool Scores

• open_questionsA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral context: the tool returns a list of open items and hints that these are places where implementations diverge. However, no annotations are provided, so the description carries the burden. It does not state whether the list is static or generated, or any caching/rate limits, but for this kind of read-only tool, it is adequate. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short paragraphs, each earning its place: purpose, timing, action on hit, consequence. No fluff, front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool, the description fully explains when and how to use it and what it returns. It doesn't describe output format, but an output schema exists, so that's not needed. It is complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so the description has little to add. The description explains what the output represents (open design decisions), adding semantic meaning beyond the empty schema. Baseline for 0 params is 4, and it meets that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists design decisions left open by the specification. It uses specific verb 'List' and resource 'design decisions', and distinguishes it from sibling tools like list_sections or get_spec by focusing on unresolved questions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: check this tool before inventing an answer, and what to do if a question is here (say 'not decided yet' and ask). It implies when not to use (when question is not listed, it may be intentionally handled elsewhere).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sectionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It clearly discloses that the tool returns a section list with summaries rather than full content, and frames itself as a navigation helper. For a simple read-only listing tool, this is sufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and every phrase earns its place. It avoids filler while adding practical usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema, the description is complete: it says what the tool returns, why to use it, and when to start with it. It sufficiently covers the low-complexity context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is effectively 100%, so no parameter explanation is needed. The baseline of 4 applies because there is no param information to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('List the sections'), the resource ('Tank Fight specification'), and the output form ('with a one-line summary of each'). It also differentiates from siblings by positioning this as the starting point for locating rules before fetching a specific part.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance: 'Start here when you don't yet know where a rule lives' and warns that 'Fetching the whole spec is rarely necessary and wastes context.' It lacks an explicit contrast with search_spec, which is a sibling that may be a better alternative when users know search terms.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_specA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds crucial context: the spec is authoritative, and it explicitly warns against inferring values from existing implementations. However, it does not mention error handling or behavior for invalid section names, so it's not fully transparent on edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with a clear one-sentence summary followed by usage guidance and a well-formatted Args section. Every sentence adds value, and the formatting makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool with an output schema, the description is complete: it explains the purpose, parameter format, and authoritative nature of the data. It appropriately references list_sections for valid section names and provides examples of dotted paths. No additional return-value details are needed given the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only declares 'section' as a string with no description, so the description provides all parameter semantics. It explains that the value can be a section name from list_sections or a dotted path, with concrete examples like 'entities.bullet'. This is essential information not available from the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Return one section of the Tank Fight specification as JSON,' which is a specific verb+resource statement. It also explains the use case ('before implementing a feature') and distinguishes itself from sibling tools by focusing on retrieving a named section of the specification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states when to use the tool ('before implementing a feature') and warns against inferring or inventing values, which provides strong context. However, it does not explicitly name alternative tools or exclusions beyond referencing list_sections for section names, so it stops short of the explicit alternative guidance found in a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mazeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that all bricks are destructible and terrain changes during play, and that border walls are indestructible. It also notes that out-of-range level values are clamped, which is a behavioral trait. However, it doesn't mention the return format or any potential side effects, but given the tool is a read operation, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening, a rationale for using the expanded tiles, and a concise Args section. Every sentence adds value, and it's appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (not shown but mentioned), the description doesn't need to explain return values. It covers the key aspects: what the tool returns, why to use it, parameter semantics, and behavioral notes. The description is complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'level' parameter (level number, out-of-range clamped) and 'ascii_map' (include a picture, useful for checking build, set false to save context). This adds meaning beyond the schema's basic types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to get a level's brick layout as rectangles and expanded 20x20 tiles. It specifies the resource (level maze) and the action (get), and distinguishes it from siblings by focusing on the brick layout, while siblings like list_sections and get_spec handle other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using the expanded tile list rather than expanding rectangles manually, explaining the risk of subtle differences. It also provides guidance on when to set ascii_map to false to save context, and mentions that border walls are separate and indestructible, which helps the agent understand the tool's output.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_specA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It explains the tool is a read-only search ('costs far less context than loading whole sections') and the behavior on empty results (spec may not cover it, check open_questions). This adds transparency about the search's scope and fallback, though it doesn't detail return format or limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise, using a short intro, a usage guideline paragraph, and a parameter explanation. Every sentence adds value, and it is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and the description covers purpose, usage, fallback behavior, and parameter semantics. The output schema exists, so return values don't need explanation. It is complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'query' parameter with examples: 'Plain words describing what you need, e.g. "friendly fire" or "spawn interval" or "health bar colour".' This adds meaningful guidance beyond the schema's bare string type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search the specification for the rule or value that answers a question.' It uses a specific verb ('search') and resource ('the specification'), and provides concrete examples of questions it answers, distinguishing it from sibling tools like list_sections or get_spec.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly guides when to use this tool: 'Use this whenever you are about to assume how the game behaves'. It contrasts with alternatives by noting that searching costs less context than loading whole sections. It also provides a clear fallback: if the search comes back empty, check open_questions() before deciding anything yourself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_levelA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals important traits: integer division with truncation, potential disagreement with floating-point calculations, and clamping behavior for out-of-range values. It also states what the result includes without exposing unnecessary details. This goes beyond a generic description and provides genuinely useful edge-case information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise, with a clear lead sentence, a paragraph explaining rationale, and a compact Args section. Every sentence earns its place—no filler or redundancy. The structure (purpose, why, related tool, parameter details) is logical and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the presence of an output schema, the description covers all necessary aspects: what it computes, why it's needed, how parameters behave, and what the output references (maze). It does not omit critical context and does not need to detail return types because an output schema is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for the 'level' parameter (0% coverage), so the description must compensate. The Args section does this effectively by explaining the parameter's meaning and its clamping behavior, which is not present in the schema. This fully addresses potential confusion and guides correct usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Resolve') and a detailed resource ('what a level number actually means: how many enemy tanks the round fields in total, how many may be alive at once, and how often they arrive'). It distinguishes itself from siblings by mentioning get_maze for maze layout and contrasting with reading a file. This is more than just a restatement of the name—it provides concrete capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool instead of alternatives: 'Levels are GENERATED from a ramp formula, not stored anywhere, so this cannot be answered by reading a file — call this tool instead of computing it yourself'. It also directs to a sibling ('call get_maze for its actual layout'). This gives clear decision guidance, exceeding basic when-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.