Server Quality Checklist

Tool Scores

• fetch_dataD
  1.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description only says it calls an external API without disclosing side effects, permissions, or idempotency. Completely insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very short but not informative. It is concise but the content is a placeholder message, not valuable instructions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no output schema and only 2 parameters; yet description fails to explain what the tool does operationally. Inadequate for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters (method and endpoint) with descriptions. Description adds no additional meaning, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description is vague: 'Placeholder for a tool that calls an external API.' It does not specify what data is fetched, from which source, or the tool's actual role. No differentiation from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives. No mention of context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• process_orderC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behaviors like idempotency, side effects, and error specifics. It only mentions 'error handling' without details, leaving the agent uninformed about consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but lacks essential content. It is not verbose, but its brevity sacrifices clarity, earning a mid-range score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 params, nested objects, no output schema), the description is insufficient. It omits the tool's purpose, expected output, and error handling specifics, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%, with most parameters already described. The description adds no extra meaning beyond what the schema provides, so it meets the baseline but doesn't compensate for the 25% gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description only highlights schema features (nested objects, enums) but fails to state the tool's action—processing an order. This is vague and does not help an agent understand the tool's primary function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage context or alternatives are provided. The description does not indicate when to use this tool over siblings like 'fetch_data' or 'manage_file'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It states read/write capability and a configurable base directory, but fails to mention whether operations are destructive, reversible, or require specific permissions. No side effects or safety details are provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose. However, the word 'demonstrates' is filler and the phrase 'configurable base directory' could be clearer.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no output schema, and no annotations, the description is insufficient. It does not explain how to use the action/content dependent parameters, nor does it describe the return value. The mention of a configurable base directory is ambiguous and not resolved in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds little beyond the schema: 'reading and writing' and 'configurable base directory' do not clarify parameter semantics like the relationship between content and write action, or what 'relative' means for filePath.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads and writes local files within a configurable base directory, distinguishing it from unrelated sibling tools like fetch_data or hello_world. However, the word 'demonstrates' slightly undermines confidence, and the mention of a configurable base directory is not reflected in the schema parameters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, scenarios, or exclusions. Sibling tools are available but without any comparative context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• long_running_taskB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only mentions 'async' and 'simulates' but does not disclose critical behavioral traits such as cancellation behavior, error handling, resource consumption, or whether the task runs in the background or blocks.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that effectively conveys the core purpose. Could be slightly more structured (e.g., separating purpose from behavior), but it is concise and front-loaded with the key idea.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (2 params, no output schema). The description mentions async behavior but lacks details on return values, error responses, or how the result is retrieved. Given the demo nature, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions (100% coverage). The description adds no extra semantics beyond 'simulates long-running operation'—it neither explains how taskName relates to the simulation nor provides additional context for durationMs beyond the schema's min/max/default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is an async simulation of a long-running operation, with concrete examples like data processing and report generation. It distinguishes itself from siblings by focusing on async behavior and duration simulation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like fetch_data or process_order. The description does not specify prerequisites, ideal contexts, or situations where this tool should be avoided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hello_worldA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that it returns a personalized greeting, but does not specify if it has side effects, idempotency, or other behavioral traits. For a simple greeting tool, this is acceptable but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the purpose and action. No unnecessary words, every sentence is relevant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema, no complex behavior), the description adequately covers input and output. It is complete for its level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for one parameter, so baseline is 3. The description adds value by clarifying the output behavior (returns a personalized greeting), which goes beyond the schema's parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a greeting tool that takes a name and returns a personalized greeting. This verb+resource specification distinguishes it from sibling tools like fetch_data or long_running_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating its function, but does not explicitly state when to use it over alternatives or provide any conditions. It is adequate but lacks guidance for an agent to choose between siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.