Server Quality Checklist

Tool Scores

• hello_worldC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions the tool returns a greeting and has an optional delay, but lacks details on behavioral traits such as whether it's read-only, if it has side effects, rate limits, or error handling. The description doesn't contradict annotations (none exist), but it's insufficient for a tool with parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with a brief overview and structured parameter/return sections. It's front-loaded with the purpose, but the formatting with indentation and bullet-like points could be more streamlined for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters, no output schema, no annotations), the description is somewhat complete but has gaps. It explains parameters and returns, but lacks behavioral context and usage guidelines. For a simple tool, it's adequate but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'name' is for greeting and 'delay' is an optional delay in seconds, which clarifies beyond the schema's basic titles. However, it doesn't cover defaults (e.g., name defaults to 'World') or constraints, leaving some gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool returns a greeting, which is a clear purpose, but it's vague about what 'hello world' entails beyond that. It doesn't distinguish from siblings like get_version or system_info, which might also return simple outputs. The phrase 'simple hello world tool' is somewhat tautological with the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like get_version or system_info. The description implies it's for greetings, but there's no explicit context for when it's appropriate or any exclusions mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_versionB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states what the tool does without disclosing behavioral traits. It doesn't mention whether this is a read-only operation, if it requires authentication, rate limits, or what format the version information returns, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and behavioral unknowns, the description is incomplete. It doesn't address what the return value looks like, potential errors, or operational context, leaving the agent with insufficient information for confident use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, earning a high baseline score for not adding unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('server version information'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'system_info' which might also provide version-related data, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'system_info' or 'hello_world'. The description offers no context about appropriate use cases or exclusions, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• system_infoB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'Get[s]' information, implying a read-only operation, but doesn't specify if it requires permissions, has rate limits, returns real-time vs. cached data, or what format the output takes. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('Get basic system information.') that is front-loaded and wastes no words. It directly communicates the core purpose without unnecessary elaboration, making it highly concise and well-structured for its simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is minimally complete. It states what the tool does but lacks details on behavioral traits, output format, or usage context. This is adequate for a basic tool but leaves gaps that could hinder an agent's understanding in more complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% description coverage, so no parameter documentation is needed. The description doesn't add parameter details beyond the schema, which is appropriate here. A baseline of 4 is applied as it adequately handles the lack of parameters without introducing confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get basic system information' clearly states the verb ('Get') and resource ('basic system information'), making the purpose immediately understandable. It distinguishes from siblings like 'get_version' (which likely returns version-specific data) and 'hello_world' (a test endpoint) by focusing on general system details. However, it doesn't specify what 'basic' entails (e.g., OS, memory, CPU), keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_version'. The description implies it's for retrieving system information but doesn't clarify if 'get_version' is a subset or a different type of data, nor does it mention any prerequisites or exclusions. This leaves the agent without explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.