Server Quality Checklist

Tool Scores

• sized_echoA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the basic behavior (returns filler of given size) but does not specify the nature of the filler (e.g., random, repeating), whether size is exact, or any performance characteristics. The read-only and safe nature is implied but not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, followed by the purpose. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description covers the basic function and purpose. However, without an output schema, it omits details about the exact format of the returned filler, which could be relevant for buffer probing. Still adequate for a test utility.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the size_kb parameter with range 0-8192. The description only repeats the parameter name without adding new meaning, so it does not surpass the baseline of 3 given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a specified number of kilobytes of filler, and its purpose of probing buffer/message size effects distinguishes it from siblings like echo and ping. The verb 'returns' and the resource 'filler' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The second sentence explicitly says the tool is 'for probing buffer and message size effects', providing clear context for when to use it. It doesn't mention alternatives or exclusions, but the purpose is clearly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden. It reveals the return payload (fixed string and sequence number) and implies a lightweight, safe operation, but doesn't explicitly state side-effect freedom or auth needs. Adequate for a simple ping, but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the core concept. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and lack of required input, the description adequately covers purpose and return behavior. It could mention sequence number semantics (e.g., increments per call), but this is not essential for a basic ping.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema confirms this. With no parameters, the baseline is 4, and the description correctly omits parameter details as they are unnecessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a minimal round trip and returns a fixed string and sequence number. It distinguishes from siblings like echo and slow_echo by emphasizing 'smallest possible round trip'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when minimal latency is desired via 'smallest possible round trip'. It doesn't explicitly name alternatives, but the sibling context makes the intended use clear, earning a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• echoA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. 'Returns its text argument unchanged' fully discloses the behavior with no hidden side effects or prerequisites. This is a complete and honest description for a simple echo tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, followed by a practical usage hint. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and full schema coverage, the description is complete: it states the core function and gives a relevant use case. No output schema is needed, and the sibling tool names make the context clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the single 'text' parameter with 'Text to return unchanged' (100% coverage). The tool description adds no additional semantic information beyond paraphrasing the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Returns its text argument unchanged', which is a specific verb and resource. However, it does not explicitly name sibling tools or differentiate from ping, sized_echo, or slow_echo, so it lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context with 'Use a different text per chat window to tell the windows apart in the capture', indicating a specific use case. However, it does not explicitly discuss when not to use this tool or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• slow_echoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the delay behavior and adds contextual meaning about isolating slow handlers. For a simple read-only echo tool, this is sufficient transparency; no side effects or permissions are mentioned but none are expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the core function, the second adds rationale. No wasted words, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description fully explains what it does, how it works, and why it exists. The behavior is completely disclosed and the schema covers inputs, making this a complete package.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters with clear descriptions (text and delay_ms). The tool description confirms the role of text and delay but does not add meaning beyond what the schema already provides. Baseline is appropriate given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: returning the text argument after a delay. It distinguishes itself from siblings like echo and sized_echo by highlighting the delay feature. The phrase 'Separates a slow handler from a lost message' provides additional purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case (simulating slow handlers) but does not explicitly compare with siblings or state when not to use it. The context is clear enough for an agent to infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.