Server Quality Checklist

Tool Scores

• discover_agentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'discover and register' which implies a write operation (registration), but doesn't clarify permissions, side effects, or what 'register' entails (e.g., updating a cache, adding to a database). This is a significant gap for a tool that appears to modify system state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part earns its place, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's apparent complexity (involving discovery and registration of CLI agents), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what 'register' means, what the output looks like, or how it interacts with system state, leaving critical gaps for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'force_refresh' well-documented in the schema. The description doesn't add any parameter-specific details beyond what the schema provides, so it meets the baseline of 3 for adequate coverage without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('discover and register') and resource ('available CLI agents on the system'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'list_agents', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_agents' or 'get_routing_guidance'. There's no mention of prerequisites, typical use cases, or exclusions, leaving the agent with little context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_agentsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists agents and availability status but doesn't cover critical aspects like whether this is a read-only operation, potential rate limits, authentication requirements, or what the output format looks like. This leaves significant gaps for a tool that presumably interacts with agent data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It is front-loaded and appropriately sized for a simple tool, earning a perfect score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'availability status' entails, how the list is formatted, or any behavioral traits like safety or performance. For a tool that likely returns structured data about agents, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so there are no parameters to document. The description appropriately doesn't mention any parameters, which aligns with the schema. A baseline of 4 is applied since no parameters exist, and the description doesn't add unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all registered agents and their availability status'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'discover_agents' or 'get_routing_guidance', which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'discover_agents' or 'get_routing_guidance'. It lacks any context about prerequisites, timing, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_routing_guidanceB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns guidance without execution, which is helpful. However, it doesn't address important behavioral aspects like whether this requires authentication, has rate limits, what happens with invalid queries, or the format/structure of the guidance returned. For a tool with zero annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and well-structured in a single sentence that clearly communicates the core functionality ('Get routing guidance for a task'), specifies the outputs ('returns which agent should handle it and the exact CLI command to run'), and adds crucial behavioral context ('guidance only, no execution'). Every word earns its place with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description provides adequate basic information about what the tool does but lacks completeness. It doesn't describe the format of the guidance returned, error conditions, or operational constraints. For a tool that presumably returns structured routing decisions, more context about the output would be helpful, though the description meets minimum viable standards.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the single parameter 'query' well-documented in the schema as 'The task query to get routing guidance for'. The description adds no additional parameter information beyond what the schema provides. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't enhance parameter understanding but doesn't need to compensate for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get routing guidance for a task' with specific outputs ('which agent should handle it and the exact CLI command to run'). It distinguishes from potential siblings by specifying this is 'guidance only, no execution'. However, it doesn't explicitly differentiate from 'discover_agents' or 'list_agents' which likely list agents rather than provide routing decisions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating it provides 'guidance only, no execution', suggesting this tool should be used when you need routing advice rather than actual task execution. However, it doesn't explicitly state when to use this versus the sibling tools 'discover_agents' or 'list_agents', nor does it provide any exclusion criteria or prerequisites for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.