Server Quality Checklist

Tool Scores

• create_itineraryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'creates' implying a write operation, but doesn't mention permissions, side effects, rate limits, or what the output looks like (e.g., format, success indicators). This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and appropriately sized, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain the return values, error conditions, or behavioral traits like idempotency or data persistence. Given the complexity of creating a personalized itinerary, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information beyond what's in the schema, which has 100% coverage with clear descriptions for all 6 parameters. The baseline score of 3 is appropriate as the schema does the heavy lifting, but the description doesn't enhance understanding of how parameters interact or affect the itinerary creation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('creates') and resource ('personalized travel itinerary'), specifying it's based on user preferences. However, it doesn't differentiate from sibling tools like 'optimize_itinerary' which might also create or modify itineraries, so it's not fully distinctive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'optimize_itinerary' or other siblings. It mentions 'based on user preferences' but doesn't specify prerequisites, exclusions, or comparative contexts, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_accommodationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions searching but doesn't disclose behavioral traits such as whether results are paginated, if authentication is required, rate limits, or what the output format looks like. For a search tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a search operation with 4 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns, how results are structured, or any behavioral constraints, leaving the agent with insufficient information for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal meaning beyond the input schema, which has 100% coverage with clear descriptions for all parameters. It implies location-based searching but doesn't provide additional context like search radius or result limits. With high schema coverage, the baseline score of 3 is appropriate as the schema does most of the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('searches') and resource ('accommodation options'), and specifies the scope ('in a specified location'). It doesn't explicitly distinguish from sibling tools like 'create_itinerary' or 'search_attractions', but the focus on accommodations is clear enough for basic differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_itinerary' or 'search_attractions', nor does it mention prerequisites or exclusions. It merely states what the tool does without context for selection among available options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• optimize_itineraryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it 'optimizes based on specified criteria' without disclosing behavioral traits like whether it modifies the original itinerary, requires specific permissions, has rate limits, or what the optimization entails (e.g., reordering, time adjustments). This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loaded with the core action. Every word earns its place, making it appropriately sized and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity as a mutation operation with no annotations and no output schema, the description is incomplete. It fails to explain what optimization does, what the return values are, or behavioral risks, leaving the agent with insufficient context for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('itineraryId' and 'optimizationCriteria'). The description adds no additional meaning beyond what the schema provides, such as examples of criteria or format details, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('optimizes') and resource ('an existing itinerary'), specifying it works on existing itineraries rather than creating new ones. However, it doesn't differentiate from sibling tools like 'create_itinerary' or 'search_attractions' beyond the core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_itinerary' or 'search_attractions', nor does it mention prerequisites such as needing an existing itinerary ID. It lacks explicit when/when-not instructions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_attractionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Searches for') but doesn't describe what the search returns (e.g., list format, pagination), potential limitations (e.g., rate limits, data sources), or error conditions. For a search tool with zero annotation coverage, this lack of behavioral details is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without any fluff. It is front-loaded with the core action and resource, making it easy to parse. Every word earns its place, achieving optimal conciseness for this level of detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search function with 3 parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain what the output looks like (e.g., list of attractions with details), behavioral aspects, or integration with sibling tools. This leaves significant gaps for an agent to understand how to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear parameter descriptions (e.g., 'Location to search attractions', 'Search radius in meters'). The description adds no additional parameter semantics beyond what the schema provides, such as examples or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Searches for') and resource ('attractions and points of interest'), and specifies the scope ('in a specified location'). It distinguishes itself from siblings like 'get_accommodations' or 'get_transport_options' by focusing on attractions. However, it doesn't explicitly differentiate from potential overlapping tools (none present), keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or how it relates to sibling tools like 'create_itinerary' or 'optimize_itinerary', which might be used in conjunction. Without any usage context, this is a significant gap in helping the agent select the right tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transport_optionsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'retrieves' implies a read-only operation, the description doesn't mention important behavioral aspects like rate limits, authentication requirements, response format, whether results are cached, or what happens with invalid inputs. For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without any wasted words. It's appropriately sized for a straightforward retrieval tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with 100% schema coverage but no annotations and no output schema, the description is minimally adequate. It states what the tool does but lacks important context about behavioral characteristics, usage guidelines relative to siblings, and what the return value contains. The absence of output schema means the description should ideally hint at what's returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters with clear descriptions. The description adds no additional parameter semantics beyond what's in the schema - it mentions 'between two points' which corresponds to origin/destination parameters, but provides no extra context about format, constraints, or usage beyond the schema's descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('retrieves') and resource ('available transportation options'), and specifies the scope ('between two points'). However, it doesn't differentiate this tool from potential sibling tools like 'optimize_itinerary' that might also involve transportation options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'create_itinerary' and 'optimize_itinerary' that might involve transportation planning, there's no indication of when this specific retrieval tool is appropriate versus those more comprehensive tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.