Server Quality Checklist

Tool Scores

• write_fileC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It mentions 'automatic code quality check' but does not clarify if the tool actually writes the file, what side effects occur, or what the return value is. The name and description conflict.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise but not optimally clear due to ambiguity about the tool's actual function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's name and the lack of output schema or annotations, the description fails to fully inform the agent about the tool's behavior, leading to potential misuse.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with basic parameter descriptions. The tool description adds no extra meaning to the parameters, earning a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is ambiguous: it says 'before writing a file, you must call this tool' (implying a pre-check), but the tool name 'write_file' suggests it performs the write. It does not explicitly state the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear context for when to use the tool (before writing a file), but lacks guidance on alternatives or when not to use it, given sibling tools like 'run_command'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_commandA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It mentions 'automatically adds protection' but lacks detail on what protection entails (e.g., sandboxing, permission checks, side effects). More transparency is needed for safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is efficient and front-loaded. However, it is slightly too terse and could include more useful information without sacrificing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters, no output schema, no annotations, and sibling tools are different, the description is minimal. It covers the essential purpose but lacks details on behavior, return values, and edge cases, making it less complete for a tool that appears to be a mandatory safety wrapper.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so both parameters are described in the input schema. The description adds no additional meaning beyond the schema, such as parameter usage or constraints. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool must be called before executing commands and automatically adds protection. It distinguishes from sibling tools (respond_to_user, write_file, complete_task) which serve different purposes. However, the exact nature of 'protection' and what 'commands' refers to is vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs the agent to call this tool before executing commands, providing a clear usage context. It does not specify when not to use it or mention alternatives, but the directive is strong and unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• respond_to_userA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It mentions 'automatic check and repair', which hints at additional logic, but does not explain what checks or repairs occur, nor side effects or idempotency. This leaves gaps for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys the mandatory nature and the action. It is front-loaded with the critical instruction. It could be slightly more structured but is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one parameter and no output schema, the description is minimal. It does not explain what happens after the tool is called (e.g., confirmation, error handling) or elaborate on the 'automatic check and repair'. For a core interaction tool, more detail would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'message' parameter. The tool description adds no extra semantic meaning beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'AI must call this tool before replying to the user'. It specifies the action (respond to user) and the resource (user message), and distinguishes from siblings (write_file, run_command, complete_task) which are different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'before replying to the user'. It implies mandatory usage, so no alternatives are needed. It does not provide explicit exclusions, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• complete_taskA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It indicates the tool performs verification, but does not disclose what happens upon success or failure, nor any side effects. Adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that conveys the essential information without any wasted words. It is front-loaded with the key requirement and is immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers the tool's purpose and usage order but does not explain return values or verification details. It is minimally adequate for a simple tool with 2 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters. The tool description adds no additional meaning about the parameters beyond what schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it must be called before marking a task as complete to verify it is truly complete. The verb 'verify' and resource 'task' are specific, and it distinguishes from sibling tools like respond_to_user, write_file, and run_command.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'must be called before marking task as complete', providing clear when-to-use guidance. It does not mention alternatives or when not to use, but sibling tools are sufficiently different that exclusions are implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.