Server Quality Checklist

Tool Scores

• get_sectionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool gets content, implying a read operation, but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, returns structured or unstructured data, or handles errors. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste—it directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool and front-loaded with essential information, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (implied to fetch content), lack of annotations, no output schema, and 0 parameters, the description is incomplete. It doesn't explain what 'complete content' includes (e.g., text, metadata), how the section is specified, or what the return format is. For a tool that likely involves data retrieval, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameters need documentation. The description adds value by implying that a 'specific rules section' is targeted, but doesn't specify how that section is identified (e.g., by ID or name). Since there are no parameters, a baseline of 4 is appropriate, as the description provides some context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('complete content of a specific rules section'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_sections' (which likely lists sections) or 'search_rules' (which likely searches across rules), leaving some ambiguity about when to choose this tool over others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_sections' or 'search_rules'. It implies usage when you need content of a specific section, but doesn't specify prerequisites (e.g., how to identify the section) or exclusions (e.g., not for listing multiple sections). This lack of context makes it harder for an agent to select correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_rulesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the action ('Search') but fails to describe key traits such as whether this is a read-only operation, how results are returned (e.g., pagination, format), or any limitations (e.g., rate limits, authentication needs). This leaves significant gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality without any wasted words. It directly conveys the tool's purpose in a clear and structured manner, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the search returns (e.g., rule text, metadata, or identifiers), how results are structured, or any behavioral nuances. For a search tool with no structured data support, this leaves the agent with insufficient information to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately adds semantic context by specifying the search scope ('Roller Derby rules') and method ('by keyword'), which goes beyond the empty schema. A baseline of 4 is applied since no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Search') and resource ('Roller Derby rules'), and indicates the search mechanism ('by keyword'). However, it doesn't explicitly differentiate from sibling tools like 'get_section' or 'list_sections' in terms of scope or functionality, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_section' or 'list_sections'. It lacks context about scenarios where keyword search is preferred over section-based retrieval, leaving the agent without explicit usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sectionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the action but doesn't disclose behavioral traits such as whether this is a read-only operation, potential rate limits, authentication needs, or what format the list returns. This leaves significant gaps for a tool with no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all available rules sections') with zero wasted words. It's appropriately sized for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate but lacks completeness. It doesn't explain return values or behavioral context, which is a gap since no output schema exists to cover that. However, for a basic list operation, it meets the minimum viable threshold.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema description coverage, the schema already fully documents the input (none). The description adds no parameter information, which is appropriate here. A baseline of 4 is given since no parameters exist, and the description doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('rules sections') with the scope 'all available', making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'search_rules' or 'get_section', which would require more specific scope or filtering details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'search_rules' or 'get_section'. The description implies a comprehensive listing without filtering, but this isn't stated explicitly, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.