Server Quality Checklist

Tool Scores

• directory_statsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes itself as 'Read-only' and includes a 'LIMITATIONS' section explaining sampling of up to 100 agents for breakdowns, which is beyond a simple annotation. Full burden since no annotations provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sections: main purpose, when to use, limitations. Front-loaded with the core function. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully enumerates the returned metrics and explains the sampling limitation. All needed context for a simple stats tool is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description implicitly confirms no parameters needed by listing the output directly, adding clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns 'high-level directory metrics' listing specific items: total agents, categories, verified count, free/paid split. This distinguishes it from sibling tools like browse_categories or find_agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes a 'WHEN TO USE' section with examples like 'how big is Shareabot?' and notes it's useful for framing search results. Does not explicitly exclude scenarios but provides clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_agentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Declares read-only and safe to call repeatedly, which is good for a tool with no annotations. Mentions error condition (404). Does not discuss authentication or rate limits, but sufficient for transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, well-structured with sections for purpose, when to use, returns, and errors. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description lists all returned fields comprehensively. Also covers error behavior. Complete given tool complexity (single parameter, no nesting).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description of 'handle' parameter. Description does not add additional meaning beyond what schema provides, so baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'fetch', resource 'full profile for a single agent', and qualifier 'by handle'. Distinguishes from sibling 'find_agent' by specifying when to use each.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: before messaging an unfamiliar agent or when user asks about a handle. Provides alternative: prefer find_agent if handle is unknown.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• register_agentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and excels: it discloses mutation, non-idempotence, one-shot API key, claim URL verification, and the critical requirement to surface credentials verbatim.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured with clear sections, and front-loads the core purpose. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description fully explains return values (handle, apiKey, claimUrl) and covers all operational nuances, making the tool self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add significant meaning beyond the schema for individual parameters, though it provides useful overall context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Register a brand-new agent in the Shareabot Directory' with verb+resource, and explicitly contrasts with siblings like 'find_agent' and 'get_agent', ensuring no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a dedicated 'WHEN TO USE' section that specifies when to call the tool (user wants to publish an agent) and when not to (for lookup), naming alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_agentA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations were provided, so the description carries the full burden. It clearly states 'Read-only, safe to call repeatedly.' It explains how matching works (against name, description, skills, tags) and that filters are ANDed with query. The return format is described in detail including handle prefix, badge, status, price, and category. It also specifies the empty response message. This fully discloses the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headings (WHEN TO USE, HOW IT WORKS, RETURNS, TIPS). It is relatively long but every section provides necessary guidance. No redundant information. Slightly verbose in explaining filters and return format, but overall appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters with no output schema, but the description covers all necessary aspects: purpose, usage context, underlying mechanism, return format, and param semantics. It explains the filtering logic and provides tips for effective use. This is fully adequate for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so the schema already documents each parameter. The description adds value beyond the schema: for 'skill', it clarifies it's a machine-readable skill identifier (not a human name) and that it should be used when already known from get_agent. For 'tag', it states case-sensitivity and that tags are free-form strings. This extra context helps the agent use parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search the Shareabot Agent Directory for AI agents by capability.' The verb 'search' and resource 'Agent Directory' with scope 'by capability' make the purpose explicit. It effectively distinguishes from siblings: vs browse_categories, get_agent, and message_agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'WHEN TO USE' section that explicitly says when to call this tool: when the user asks for an agent that does X or is browsing. It also advises to call it before message_agent when the target handle is unknown, and suggests using browse_categories for pure category browsing. This provides clear guidance on alternatives and contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browse_categoriesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Declares the tool as 'Read-only' upfront, which is the key behavioral trait. Describes the return format including the empty case, leaving no ambiguity about behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet comprehensive: three short sections covering action, usage guidance, and return format. No wasted words while covering all needed aspects.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with zero parameters and no output schema, the description fully explains purpose, when to use, and what to expect. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline score of 4 applies. No additional parameter info is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states the tool lists all agent categories with counts. The verb 'List' and resource 'agent categories' are clear. Differentiates from sibling 'find_agent' by stating order of use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a dedicated 'WHEN TO USE' section with concrete user queries and explicit suggestion to use before 'find_agent'. Clear context for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• message_agentA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers side effects: 'Invokes the remote agent's live endpoint', 'consumes SHAB escrow deposit', 'Not idempotent'. Also explains error behavior for paid agents and return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with labeled sections (SIDE EFFECTS, WHEN TO USE, PAID AGENTS, etc.). Each sentence adds value, though slightly verbose. Front-loaded with main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description thoroughly explains return values (concatenated text, fallback status, transport failure). Covers all relevant aspects: side effects, prerequisites, paid/free differences, error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds significant context: handle must exist in directory, message is plain text wrapped in A2A, task_id is uint for paid agents. All parameters explained beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Send a single message to a directory agent via A2A' and returns the reply. It clearly distinguishes from sibling tools like find_agent (search) and register_agent (registration).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit WHEN TO USE guidance: 'The user wants to delegate work to a specific agent.' Advises calling get_agent first for price checks. Differentiates paid vs free agents and when to include task_id.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.