Server Quality Checklist

Tool Scores

• YOUR_TOOL_NAMED
  1.1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, but 'YOUR_TOOL_DESCRIPTION' offers no information about the tool's behavior, such as whether it performs read/write operations, requires authentication, has side effects, or handles errors. It fails to describe any behavioral traits beyond the placeholder text.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While the description is technically concise with a single word, it is under-specified rather than efficiently informative. The placeholder 'YOUR_TOOL_DESCRIPTION' does not earn its place as it provides no useful content, making it ineffective despite its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 1 parameter with no schema descriptions, no annotations, no output schema, and sibling tools, the description is completely inadequate. It fails to address any contextual aspects, such as the tool's role among siblings or how it complements other tools, leaving the agent with insufficient information to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 required parameter with 0% description coverage, and the description 'YOUR_TOOL_DESCRIPTION' adds no meaning or context about the 'query' parameter. It does not explain what the query represents, its format, or how it influences the tool's operation, leaving the parameter undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'YOUR_TOOL_DESCRIPTION' is a tautology that merely restates the tool name 'YOUR_TOOL_NAME' without providing any meaningful information about what the tool actually does. There is no verb, resource, or specific functionality mentioned, making it impossible to understand the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool, when not to use it, or how it relates to sibling tools like 'roll_dice' and 'web_search'. There are no contextual cues, prerequisites, or alternatives mentioned, leaving the agent with no usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• roll_diceC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Roll') but doesn't explain key behaviors: whether this is a deterministic or random process, what the output format might be (e.g., individual rolls, totals, or details), or any constraints like rate limits. For a tool with no annotation coverage, this leaves significant gaps in understanding how it operates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's front-loaded with the core action ('Roll the dice'), making it easy to scan. However, its brevity contributes to underspecification rather than optimal clarity, as it lacks details needed for full understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (involving dice notation and multiple rolls), no annotations, no output schema, and low schema description coverage, the description is incomplete. It doesn't explain what the tool returns (e.g., roll results), how errors are handled, or the semantics of parameters. For a tool that likely produces varied outputs, this leaves the agent with insufficient context to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'notation' but doesn't define what that entails (e.g., standard dice notation like '3d10+2'). It doesn't address 'num_rolls' at all, leaving its purpose unclear. With two parameters and no schema descriptions, the description adds minimal semantic value beyond hinting at one parameter's role.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('Roll') and the resource ('dice'), but it's vague about what 'with the given notation' means. It doesn't specify what the dice notation entails (e.g., '2d6' for two six-sided dice) or how it differs from potential siblings like 'web_search' or 'YOUR_TOOL_NAME', which are unrelated. This provides a basic purpose but lacks specificity and differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives. It doesn't mention any context for rolling dice (e.g., for games, simulations, or random generation) or exclusions (e.g., not for mathematical calculations). With sibling tools like 'web_search' that serve different purposes, this lack of comparative guidance leaves the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'Search the web' but doesn't specify aspects like rate limits, authentication needs, result format, or potential side effects (e.g., whether it accesses external APIs or has usage constraints). This leaves significant gaps in understanding how the tool behaves beyond its basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's front-loaded with the core action ('Search the web'), making it easy to parse. Every part of the sentence contributes essential information, achieving optimal conciseness for such a straightforward tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (web search with external dependencies), no annotations, no output schema, and low schema coverage, the description is insufficiently complete. It lacks details on behavioral traits, result handling, error conditions, or usage constraints, which are critical for an AI agent to invoke this tool effectively in varied contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It adds meaning by explaining that the 'query' parameter is used to search for information on the web, which clarifies its purpose beyond the schema's basic type definition. However, it doesn't provide details on query formatting, length limits, or examples, resulting in only partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search the web') and the resource ('information about the given query'), making the purpose immediately understandable. It doesn't distinguish from sibling tools like 'roll_dice' or 'YOUR_TOOL_NAME', but those are unrelated, so differentiation isn't critical here. The description avoids tautology by specifying what kind of search it performs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or in what context it's appropriate. It simply states what it does without indicating any prerequisites, limitations, or scenarios where it might be preferred over other tools. This lack of usage context leaves the agent with minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.