Server Quality Checklist

Tool Scores

• get_agentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the return data includes 'description, README, capabilities, stars, forks, languages and more,' which adds some context about output. However, it doesn't cover critical aspects like whether this is a read-only operation, error handling for invalid slugs, rate limits, or authentication needs, leaving significant gaps for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by details on return values. It's efficient with two sentences and no redundant information, though it could be slightly more structured by separating usage context from output details. Overall, it's appropriately sized and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is moderately complete. It covers the purpose and output data, but lacks behavioral details like error cases or operational constraints. Without annotations or an output schema, it should provide more context on what 'full details' entail and potential limitations, making it adequate but with noticeable gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'slug' parameter well-documented as 'Agent slug — e.g. 'playwright', 'ollama', 'claude-code', 'mem0'.' The description adds minimal value by restating that the slug is 'name as kebab-case' and used to identify the agent, but doesn't provide additional semantics beyond what the schema already covers. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get full details for a specific agent by its slug.' It specifies the verb ('Get'), resource ('agent'), and identifier method ('by its slug'), making the action clear. However, it doesn't explicitly differentiate from sibling tools like 'list_agents' or 'search_agents', which likely handle multiple agents rather than a single one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need detailed information about a specific agent, as indicated by 'specific agent by its slug.' However, it doesn't provide explicit guidance on when to use this tool versus alternatives like 'list_agents' or 'search_agents,' nor does it mention any prerequisites or exclusions. The context is clear but lacks comparative direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_agentsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'browse top agents by category,' which hints at a read-only operation, but doesn't explicitly state safety (e.g., non-destructive), rate limits, authentication needs, or what the output looks like (e.g., pagination, format). For a tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste. The first sentence states the core purpose, and the second provides usage guidance. It's front-loaded and efficiently structured, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose and hints at usage, but lacks details on behavioral traits (e.g., safety, output format) that are crucial since annotations are absent. It meets minimum viability but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all parameters well-documented in the schema (type, sort, limit). The description adds minimal value beyond the schema, mentioning 'optionally filtered by type' and 'browse top agents by category,' which loosely relates to the 'type' and 'sort' parameters but doesn't provide additional syntax or usage details. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List agents from A2ASearch, optionally filtered by type.' It specifies the verb ('List'), resource ('agents'), and source ('A2ASearch'), and mentions optional filtering. However, it doesn't explicitly differentiate from sibling tools like 'search_agents' beyond implying this is for browsing rather than searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context: 'Use this to browse top agents by category.' This implies it's for general browsing rather than targeted searches, but it doesn't explicitly state when to use this vs. 'search_agents' or 'get_agent', nor does it mention any prerequisites or exclusions. The guidance is helpful but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_agentsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the search behavior and return format, but lacks details on rate limits, authentication needs, pagination, or error handling. It adds basic context but does not fully compensate for the absence of annotations, leaving gaps in behavioral understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence and adds return details in the second. Both sentences earn their place by providing essential information without redundancy or fluff, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with filters), no annotations, and no output schema, the description is reasonably complete. It covers the purpose, scope, and return format, but could improve by addressing behavioral aspects like rate limits or error cases. It's adequate but has minor gaps in full contextual coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description does not add any additional meaning beyond what the schema provides (e.g., it doesn't explain parameter interactions or provide examples beyond the schema's descriptions). Baseline score of 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search'), the target resource ('A2ASearch directory for AI agents, MCP servers, CLI tools and agent skills'), and the return format ('Returns name, description, type, stars, GitHub URL and capabilities for each result'). It distinguishes from sibling tools like 'get_agent' (likely fetches a single agent) and 'list_agents' (likely lists all without search) by specifying search functionality with filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching the directory with a query and optional filters, but does not explicitly state when to use this tool versus alternatives like 'list_agents' (e.g., for browsing vs. targeted search) or 'get_agent' (e.g., for specific agent details). It provides some context but lacks explicit guidance on exclusions or comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.