Server Quality Checklist

Tool Scores

• roll_diceC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action but lacks details like whether this is a random simulation, if it has side effects, error handling, or output format. For a tool with no annotations, this is insufficient to inform the agent about its behavior beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the main action and efficiently includes the key input, making it easy to parse. Every part of the sentence contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (simple dice rolling), two parameters, no annotations, and an output schema exists (which should cover return values), the description is minimally complete. It states what the tool does but lacks details on parameters and behavior, leaving gaps that the agent must infer from the schema and output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter details. The description mentions 'notation' but doesn't explain what it means (e.g., dice notation syntax) or the purpose of 'num_rolls'. It adds minimal semantics beyond the parameter names, failing to compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('Roll the dice') and mentions the key input ('with the given notation'), which clarifies the basic purpose. However, it's vague about what 'notation' means (e.g., dice notation like '2d6+1') and doesn't differentiate from siblings like get_stock_ohlc or web_search, which are unrelated tools. It avoids tautology by not just restating the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention any context, prerequisites, or exclusions, such as when to roll dice in a game simulation versus using other tools. It's a standalone statement with no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool searches the web but doesn't reveal any behavioral traits such as rate limits, authentication needs, result format, or potential side effects (e.g., network usage). This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly. Every part of the sentence contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (web search with 1 parameter) and the presence of an output schema (which handles return values), the description is partially complete. It covers the basic purpose but lacks behavioral details, usage guidelines, and parameter nuances. With no annotations, it should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the schema provides no semantic information. The description adds meaning by specifying that the 'query' parameter is used to search the web for information, which clarifies its purpose. However, it doesn't detail query formatting, constraints, or examples, leaving gaps in parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('the web'), specifying what the tool does. It distinguishes from siblings like 'get_stock_ohlc' and 'roll_dice' by focusing on web search functionality. However, it could be more specific about the type of web search (e.g., general information vs. specific sources).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios where web search is appropriate, nor does it exclude cases where siblings might be better (e.g., using 'get_stock_ohlc' for stock data). This lack of context leaves the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_stock_ohlcA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool retrieves data but does not specify whether it requires authentication, has rate limits, what happens if the date is invalid or the symbol doesn't exist, or if it's a read-only operation. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose and key usage details without any unnecessary words. It is front-loaded with the main action and resource, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but with an output schema), the description is mostly complete. It covers the purpose and basic usage, and the presence of an output schema means return values are documented elsewhere. However, it lacks details on error handling or behavioral constraints, which could be important for robust use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds some meaning by clarifying that 'date' is optional and defaults to the last working day, which is not evident from the schema alone (schema description coverage is 0%). However, it does not explain the format of the 'symbol' parameter (e.g., ticker format) or the 'date' parameter (e.g., YYYY-MM-DD), leaving room for ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get') and resource ('Open, High, Low, Close (OHLC) data for a stock symbol'), with precise scope ('for the last working day or specified date'). It effectively distinguishes from sibling tools (roll_dice, web_search) by focusing on financial data retrieval rather than random generation or web search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (to retrieve OHLC data for stocks, either for the last working day or a specified date). However, it does not explicitly state when not to use it or name alternatives for similar financial data queries, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.