Server Quality Checklist

Tool Scores

• tool_that_access_requestC
  2.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide substantial behavioral information (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false), so the description's burden is lower. The description adds no behavioral context beyond what annotations already declare - it doesn't mention what type of access occurs, whether authentication is needed, rate limits, or what happens when accessing requests. However, it doesn't contradict annotations either, so it meets the minimum baseline for descriptions with good annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While technically concise with only three words, this represents under-specification rather than effective brevity. The description fails to provide necessary information about the tool's purpose and usage. Every sentence should earn its place, but this single sentence doesn't provide enough value to justify its existence as a helpful description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this tool has one required parameter, annotations covering key behavioral aspects, and an output schema exists, the description is incomplete. While the output schema means the description doesn't need to explain return values, the description fails to explain what 'access the request' means in practical terms, what kind of requests are involved, or provide any operational context. For a tool with parameter requirements and behavioral implications, this description leaves too many questions unanswered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the username parameter fully documented in the schema. The description adds no parameter information whatsoever - it doesn't explain why username is required, what relationship it has to 'accessing the request', or provide any context beyond what's already in the structured schema. This meets the baseline score of 3 when schema coverage is high and description adds no parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Access the request' is essentially a tautology that restates the tool name 'tool_that_access_request' without adding meaningful clarification. It doesn't specify what type of request is being accessed, what 'access' entails (e.g., retrieve, modify, approve), or what resource is involved. While it distinguishes from unrelated siblings like get_weather, it fails to provide specific verb+resource information needed for clear purpose understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides absolutely no guidance on when to use this tool versus alternatives. There's no mention of context, prerequisites, or comparison with sibling tools like get_called_tools, get_time, or get_weather. The agent receives no information about appropriate use cases or when this tool would be preferred over other options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weatherB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide key behavioral hints (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false), covering safety and idempotency. The description adds minimal context beyond this, stating it retrieves 'current' weather, which implies real-time data but doesn't elaborate on rate limits, authentication needs, or data freshness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, rich annotations, and the presence of an output schema, the description is reasonably complete. It covers the core purpose adequately, though it lacks usage guidelines and deeper behavioral context, which are partially mitigated by the structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema fully documents both parameters (location and unit). The description adds no additional semantic context beyond implying location is required, which is already clear from the schema. Baseline 3 is appropriate when schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('current weather'), and specifies the scope ('in a given location'). However, it doesn't explicitly differentiate from potential weather-related siblings, though none are listed among the provided sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or scenarios where other tools might be more appropriate, leaving the agent with minimal context for decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_called_toolsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description doesn't add behavioral details beyond what annotations provide, but annotations are comprehensive (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false). Since annotations cover key behavioral traits, the description doesn't need to compensate, and there's no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, comprehensive annotations, and an output schema, the description is complete enough for its purpose. It could be slightly improved by clarifying what 'called tools' means, but the structured data compensates well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters, and schema description coverage is 100%, so no parameter documentation is needed. The description doesn't mention parameters, which is appropriate, earning a baseline score for this scenario.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get the list of called tools' clearly states the action (get) and resource (called tools), but it's somewhat vague about what 'called tools' means in this context. It doesn't differentiate from sibling tools like 'get_time' or 'get_weather' beyond the resource name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of context, prerequisites, or comparisons with sibling tools like 'tool_that_access_request' that might serve similar purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_timeA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide key behavioral hints (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false), so the description doesn't need to repeat these. However, it adds no additional context about rate limits, authentication needs, or specific behavioral traits beyond the basic action, resulting in a baseline score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded with a single, clear sentence that directly states the tool's purpose. There is no wasted language or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, annotations covering key behaviors, and an output schema present), the description is complete enough for basic understanding. However, it lacks any usage context or differentiation from siblings, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema description coverage, the schema fully documents the lack of inputs. The description implicitly confirms this by not mentioning any parameters, which is appropriate. A baseline of 4 is given as no parameters are present.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('current time'), making it immediately understandable. However, it doesn't distinguish itself from potential sibling tools like 'get_weather' or 'get_called_tools' beyond the resource difference, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.