Server Quality Checklist

Tool Scores

• get_viewC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions retrieving a view but lacks details on permissions, rate limits, error handling, or what 'supported view' entails. This is inadequate for a tool with 5 parameters and no annotation coverage, leaving the agent with insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no annotations) and the presence of an output schema, the description is incomplete. It doesn't address parameter meanings or behavioral traits, though the output schema may help with return values. This leaves gaps but isn't entirely inadequate, aligning with a minimum viable score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only vaguely references 'run id' and 'file path' without explaining the 5 parameters (run_id, view, selector, cursor, budget) or their relationships. This fails to add meaningful semantics beyond the bare schema, resulting in poor parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get one supported view using a numeric run id or an already-parsed file path.' It specifies the action (get), resource (view), and input types (run id or file path). However, it doesn't explicitly differentiate from sibling tools like 'parse_result' or 'search_messages', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance on when to use this tool, mentioning only the input types (run id or file path). It doesn't explain when to choose this over alternatives like 'parse_result' or 'search_messages', nor does it specify prerequisites or exclusions, leaving significant gaps in usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_messagesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but lacks behavioral details. It doesn't disclose whether this is a read-only operation, potential side effects, authentication needs, rate limits, or what the search returns. The mention of 'indexed messages' hints at a read operation but is insufficient for a mutation-aware agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core purpose and specifies input types clearly, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with 0% schema coverage and an output schema (which handles return values), the description is incomplete. It covers the 'run_id' parameter but misses others, and with no annotations, it lacks behavioral context. However, the output schema mitigates some gaps, making it minimally adequate but with clear deficiencies.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but only partially does. It explains 'run_id' accepts numeric or file path inputs, but doesn't clarify 'query', 'level', 'limit', or 'cursor' parameters. This leaves most parameters undocumented beyond schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search') and resource ('indexed messages'), and specifies the two input types ('numeric run id or an already-parsed file path'). It doesn't explicitly differentiate from sibling tools like 'get_view' or 'parse_result', but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_view' or 'parse_result'. The description mentions input types but doesn't explain context, prerequisites, or exclusions for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_resultB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions parsing and indexing but doesn't explain what 'index it' entails (e.g., creating a searchable database, storing metadata), whether it's idempotent, if it requires specific permissions, or how errors are handled. For a tool with two parameters and no annotation coverage, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'Parse a Robot Framework output.xml or RF 7.2+ output.json file and index it.' It's front-loaded with the core action and resource, with no wasted words or redundancy. Every part of the sentence contributes directly to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which reduces the need to describe return values), no annotations, and a simple input schema with low coverage, the description is minimally adequate. It states what the tool does but lacks details on behavior, parameters, and usage context. For a parsing/indexing tool, this leaves gaps in understanding how it operates and when to apply it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter semantics beyond what the input schema provides. With 0% schema description coverage, the schema only lists 'path' (string) and 'force_rebuild' (boolean, default false) without explaining their meanings. The description doesn't compensate by clarifying what 'path' refers to (e.g., file path, URL) or what 'force_rebuild' does (e.g., overwrite existing index). This meets the baseline of 3 since the schema provides some structure, but the description fails to add value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Parse a Robot Framework output.xml or RF 7.2+ output.json file and index it.' It specifies the verb (parse), the resource (Robot Framework output files), and the action (index it). However, it doesn't explicitly differentiate from sibling tools like 'get_view' or 'search_messages', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., file existence), use cases (e.g., for analysis or reporting), or exclusions (e.g., not for real-time monitoring). With sibling tools like 'get_view' and 'search_messages' available, this lack of context leaves the agent guessing about appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.