Server Quality Checklist

Tool Scores

• send_messageB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states the basic operation and the optional correlation_id. It fails to disclose important traits such as error handling (e.g., invalid handle), whether the message is asynchronous, authentication requirements, or any side effects. This is a significant gap for a communication tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the primary action front-loaded in the first sentence and the optional feature in the second. Every word serves a purpose; there is no redundancy or unnecessary detail. It is highly efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no output schema, and no annotations. The description only covers the basic action and one optional parameter. It does not explain success/failure behavior, return values, error conditions (e.g., invalid agent handle), or timing guarantees. This leaves significant gaps for an agent to correctly interpret the tool's full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage with descriptions for all three parameters. The description adds minimal extra context by explaining the correlation_id's purpose ('match a later reply'), but it doesn't enrich the 'to' or 'body' parameters beyond the schema. Given the high schema coverage, a baseline of 3 is appropriate, with only a slight increase for the added correlation_id context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('send a message'), the target ('another agent'), and the addressing method ('by handle'). It also mentions the optional correlation_id for matching replies. This directly identifies the tool's purpose and differentiates it from siblings like 'check_messages' which is for receiving.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this to send a message. However, it does not explicitly state when to use this tool versus alternatives (e.g., 'check_messages' for receiving), nor does it provide any when-not guidance or prerequisites. This leaves the agent to infer context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• presenceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the main behavior but does not disclose error handling, what 'live connection' precisely means, or any side effects. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that effectively communicates the tool's purpose with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple tool but lacks details on return format (e.g., boolean or status) and error cases. No output schema is provided, which lowers completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter 'handle' described as 'agent handle to check'. The description adds no new meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check') and the resource ('whether a given agent currently has a live connection'). It is distinct from sibling tools which focus on messaging and contacts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool's use for checking online status but does not provide explicit guidance on when to use it versus alternatives, nor does it mention any prerequisites or exclusion cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_contactsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool lists contacts and optionally filters, implying a read operation. But it does not disclose any potential behaviors like pagination, ordering, or limits, nor does it confirm idempotency. For a simple list tool, this is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two clear, front-loaded sentences. The first states the action and secondary feature (filtering), and the second provides usage context. No unnecessary words; every sentence is useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema, no nested objects), the description is sufficient. It covers the core behavior and usage. However, it could be slightly more complete by mentioning what fields are returned or any defaults, but it is not a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single optional parameter 'capability'. The description merely paraphrases the schema description ('optionally filtered by a capability tag') without adding new semantics. The schema already defines the parameter well, so the description adds minimal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists agents (contacts) with optional filtering by capability. The title reinforces this. It distinguishes from siblings like send_message (messaging) and presence (status) by specifying that it lists the agents you can message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a usage hint: 'Use this to decide who to route a task to.' This implies when to use the tool. However, it does not explicitly mention when not to use it or provide alternatives to siblings, though the guidance is clear enough given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_messagesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so description covers marking read behavior and return format. Does not mention auth or rate limits, but for a simple read operation, it is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load purpose and add necessary behavior details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one param and no output schema, the description explains behavior, return format, and statefulness. Complete for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the single parameter 'peek' at 100% coverage, and description repeats the same info. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches new messages addressed to you, with a specific verb and resource. It distinguishes from siblings like send_message and list_contacts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies repeated use ('since you last checked') and explains behavior with peek parameter. No explicit when-not or alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whoamiA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full weight. It discloses that the tool returns identity information, connection details, and a token check, which is a read-only operation. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence that captures purpose, outputs, and an additional feature (token verification) without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description completely covers what the tool does and returns. Sibling tools handle different concerns, so no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description adds value by explaining the return values beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns 'your own handle, the bus URL, and how many contacts are reachable', and additionally verifies the token. This distinguishes it from sibling tools like send_message or presence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking identity and token validity, but does not explicitly state when to use versus alternatives or provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.