Server Quality Checklist

Tool Scores

• checkC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'check messages' but doesn't disclose key behaviors: whether this is a read-only operation, if it modifies state (e.g., via auto_ack), what happens on failure, or rate limits. The example shows parameters but doesn't explain the tool's effect beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the purpose, followed by a usage example. It wastes no words, though it could be more informative. The structure is clear but minimal, earning a high score for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain return values, error handling, or the tool's role in the message system. For a tool with this complexity and lack of structured data, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds minimal value with an example using 'agent' and 'namespace', but doesn't explain parameter interactions or semantics beyond what's in the schema. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Check messages for an agent' which provides a basic verb+resource combination, but it's vague about what 'check' means (e.g., retrieve, inspect, verify). It doesn't distinguish from siblings like 'list' or 'chk' which might have similar functions. The purpose is understandable but lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage example with two parameters, but offers no guidance on when to use this tool versus alternatives like 'list' or 'chk'. There's no mention of prerequisites, exclusions, or typical scenarios. The example implies it's for checking messages, but without context for choosing among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chkC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'shortcut' but doesn't explain behavioral traits like whether it's read-only, if it modifies state, or how it handles errors. The example hints at parameters but lacks context on permissions, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences: one stating the purpose and one providing a usage example. It's front-loaded and wastes no words, though it could be slightly more informative without losing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'check' entails, return values, or error handling. Given the complexity and lack of structured data, more context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds minimal value with an example showing 'agent' and 'namespace', but doesn't explain semantics beyond what's in the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Check messages for an agent' which provides a basic purpose, but it's vague about what 'check' means (e.g., list, retrieve, inspect). It doesn't clearly distinguish from sibling tools like 'check', 'list', or 'ack', leaving ambiguity about its specific role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage example but no guidance on when to use this tool versus alternatives. With siblings like 'check', 'list', and 'ack', there's no indication of scenarios where 'chk' is preferred or what makes it a 'shortcut', leading to potential misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• broadcastC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('Send a message to all agents') but doesn't describe important behavioral traits: whether this is a synchronous or asynchronous operation, what happens if agents are offline, whether there are rate limits, what permissions are required, or what the response looks like. The example shows parameter usage but doesn't explain system behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately brief with two sentences: a clear purpose statement followed by a usage example. Every element serves a purpose, though the example could be integrated more smoothly. The structure is front-loaded with the core functionality stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a messaging tool with 3 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address key contextual questions: what format the message takes, how agents receive it, whether delivery is guaranteed, what happens on failure, or what the tool returns. The example helps but doesn't compensate for missing behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description's example shows parameter usage but doesn't add meaningful semantic context beyond what the schema provides. It doesn't explain why one would use 'namespace' for isolation or when 'from_agent' should be specified versus omitted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Send') and resource ('a message to all agents'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'send' or 'reply', but the 'to all agents' scope provides some distinction. The description avoids tautology by not just restating the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'send' or 'reply'. It includes a usage example but doesn't explain the appropriate context, prerequisites, or exclusions for using broadcast versus other messaging tools. The agent must infer usage from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cleanC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a destructive operation ('Clean up' suggests deletion) but doesn't clarify whether this is reversible, requires specific permissions, or has side effects like rate limits. The example hints at default values but lacks explicit warnings about data loss or system impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—just two sentences with zero wasted words. It front-loads the purpose and immediately follows with a practical usage example, making it easy to parse and apply. Every element serves a clear function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a destructive operation with no annotations or output schema), the description is inadequate. It doesn't explain what 'clean up' entails (e.g., deletion, archiving), what happens to the messages, or what the agent should expect as a result. For a tool that likely modifies system state, this leaves critical gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters thoroughly. The description adds minimal value by restating the parameters in the usage example without providing additional semantic context beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Clean up') and resource ('old messages'), making it immediately understandable. However, it doesn't differentiate this tool from potential sibling alternatives like 'ack' or 'check' that might also manage messages, leaving some ambiguity about its unique role in the toolset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage example but offers no guidance on when to use this tool versus alternatives like 'ack' or 'check'. There's no mention of prerequisites, exclusions, or contextual factors that would help an agent decide between this and other message-related tools in the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'PM visibility' which hints at access control, but doesn't explain what that means, whether this is a read-only operation, potential rate limits, or what happens to messages after listing. For a queue tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two parts: a clear purpose statement followed by a usage example. Both sentences earn their place by providing essential information. However, the usage example could be more integrated with the purpose statement rather than appended as a separate fragment.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a queue listing tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'pending messages' means operationally, what format the output takes, whether results are paginated beyond the limit parameter, or how 'PM visibility' affects results. The agent would need to guess about important behavioral aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description only mentions the 'namespace' parameter in the usage example, adding minimal value beyond what the schema provides. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all pending messages') and resource ('in the queue'), specifying scope with '(PM visibility)'. It distinguishes from siblings like 'send' or 'reply' by focusing on retrieval rather than creation or response. However, it doesn't explicitly differentiate from 'check' or 'chk' which might also involve queue inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage example ('list(namespace='myproject')') but offers no guidance on when to use this tool versus alternatives like 'check' or 'chk'. There's no mention of prerequisites, constraints, or comparative context with sibling tools, leaving the agent to infer appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sendC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it shows the basic action of sending a message, it doesn't describe delivery guarantees, latency, error conditions, authentication requirements, or what happens when sending to 'all' recipients. For a communication tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just one sentence that combines purpose statement with a concrete usage example. Every element earns its place - the action, target, and example parameters are all essential information presented without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a messaging tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after sending (success/failure indicators, response format), doesn't address system behavior with different recipient types, and provides no context about message delivery mechanisms or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description's usage example shows parameter names but adds no meaningful semantic context beyond what the schema provides. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send a message') and target ('to another agent'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from sibling tools like 'broadcast' or 'reply', which likely have overlapping communication functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'broadcast' or 'reply'. The usage example shows parameter syntax but offers no context about appropriate scenarios, prerequisites, or exclusions for this specific messaging tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ackB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'acknowledge/delete' implying a destructive operation, but fails to clarify permissions, side effects, or response format. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the purpose stated first followed by a concise usage example. Both sentences earn their place, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is incomplete. It lacks critical details like confirmation of deletion, error handling, or behavioral context, leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds minimal value with a usage example but no additional semantics beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('acknowledge/delete') and resource ('a message after processing'), distinguishing it from siblings like 'send' or 'reply'. It precisely defines the action and target without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'clean' or 'list'. It includes a usage example but lacks explicit context, prerequisites, or exclusions, leaving the agent to infer appropriate scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that the tool 'acks original and sends response to sender', which hints at mutation behavior, but does not disclose critical details like permissions needed, rate limits, error conditions, or what happens if the message ID is invalid. For a mutation tool with zero annotation coverage, this is a significant gap in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, with a clear purpose statement followed by a practical usage example. Every sentence earns its place without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks information on behavioral traits like error handling, response format, or system constraints. The example usage helps but does not compensate for the missing contextual details needed for reliable tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by providing an example usage with 'id', 'message', and 'namespace', but does not explain parameter semantics beyond what the schema provides. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Reply to a message') and distinguishes it from siblings like 'send' or 'broadcast' by specifying it acks the original message and sends a response to the sender. This provides a precise verb+resource combination with clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a usage example that implies when to use this tool (to reply to a specific message), but it does not explicitly state when to use it versus alternatives like 'send' or 'ack'. There is no guidance on prerequisites or exclusions, leaving usage context somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.