Server Quality Checklist

Tool Scores

• analyzeListingPhotosC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool analyzes photos but doesn't describe what the analysis entails (e.g., returns scores, detects objects), potential side effects (e.g., rate limits, data processing), or output format. This is a significant gap for a tool with no structured behavioral hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized for a simple tool, though it could be more front-loaded with key details like analysis type. There's no wasted text, earning a high score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and a simple input schema, the description is incomplete. It doesn't explain what the analysis returns, how results are structured, or any behavioral traits like error handling. For a tool that presumably performs non-trivial photo analysis, this leaves critical gaps in understanding its operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'id' documented as 'Airbnb listing ID'. The description doesn't add any meaning beyond this, such as format examples or constraints. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, though no extra value is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('analyze') and resource ('photos from an Airbnb listing'), which provides a basic understanding of purpose. However, it lacks specificity about what analysis is performed (e.g., quality assessment, content detection) and doesn't distinguish from sibling tools like 'getListingPhotos' that might retrieve photos without analysis. This makes it vague but not tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, context (e.g., after fetching listing details), or comparisons to siblings like 'airbnb_listing_details' or 'getListingPhotos'. This leaves the agent without direction on appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• airbnb_listing_detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'Get[s] detailed information,' implying a read-only operation, but doesn't address critical aspects like authentication requirements, rate limits, error handling, or what 'detailed information' includes (e.g., pricing, availability, amenities). The mention of 'direct links' hints at output behavior but is vague. For a tool with 8 parameters and no annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two short sentences that are front-loaded with the core purpose. There's no wasted text, and it efficiently communicates the basic function. However, the second sentence ('Provide direct links to the user') is somewhat vague and could be integrated more smoothly, slightly reducing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, no annotations, no output schema), the description is incomplete. It lacks details on behavioral traits (e.g., data sources, latency), output format (what 'detailed information' entails), and usage context. Without annotations or an output schema, the description should do more to guide the agent, such as explaining the return structure or common use cases, but it falls short.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are documented in the input schema (e.g., 'id' as listing ID, dates in YYYY-MM-DD format). The description adds no additional parameter semantics beyond what's in the schema, such as explaining how parameters interact (e.g., how dates affect pricing) or clarifying optional vs. required usage. Given the high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Get detailed information about a specific Airbnb listing,' which includes a specific verb ('Get') and resource ('Airbnb listing'). However, it doesn't explicitly differentiate from sibling tools like 'airbnb_search' (which likely searches multiple listings) or 'analyzeListingPhotos' (which focuses on photos). The mention of 'direct links' adds some specificity but doesn't fully distinguish it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'airbnb_search' or 'analyzeListingPhotos.' It mentions providing 'direct links to the user,' which implies a use case for sharing information, but doesn't specify prerequisites, exclusions, or contextual triggers. This leaves the agent with minimal direction on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• airbnb_searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'pagination' (implied via the 'cursor' parameter) and 'Provide direct links to the user' which adds some context about output behavior. However, it lacks critical details such as rate limits, authentication requirements, error handling, or what the search results include (e.g., listing summaries vs. full details). For a search tool with 12 parameters, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Search for Airbnb listings') and includes key features. It avoids unnecessary words, though it could be slightly more structured by separating functional aspects from output instructions. Every part earns its place, but it's not perfectly optimized for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain the return format beyond 'direct links', leaving the agent uncertain about what data is provided (e.g., listings, prices, availability). For a search tool with rich filtering options, more context on results and behavior is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 12 parameters thoroughly. The description adds minimal value by mentioning 'various filters and pagination', which aligns with parameters like 'minPrice', 'maxPrice', and 'cursor', but doesn't provide additional semantics beyond what the schema states. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Search') and resource ('Airbnb listings'), and mentions key capabilities ('various filters and pagination'). It distinguishes from sibling tools like 'airbnb_listing_details' by focusing on search rather than detailed information retrieval. However, it doesn't explicitly differentiate from 'analyzeListingPhotos' or 'getListingPhotos' which might also involve search-related functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'airbnb_listing_details' for specific listings, or how it relates to photo analysis tools. It mentions 'Provide direct links to the user' which hints at output format but doesn't clarify usage context or exclusions. No explicit when/when-not statements are present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getListingPhotosC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe how it behaves: no information on rate limits, authentication needs, error handling, or what happens if the listing ID is invalid. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized for a simple tool and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the output looks like (e.g., format of extracted URLs, whether it's a list or structured data), nor does it cover behavioral aspects like error conditions. Given the lack of structured data, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'id' clearly documented as 'Airbnb listing ID'. The description doesn't add any parameter details beyond what the schema provides, so it meets the baseline for high schema coverage but doesn't enhance understanding of parameter usage or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Extract') and resource ('photo URLs from an Airbnb listing'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'analyzeListingPhotos' which might involve more complex photo analysis rather than just URL extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'airbnb_listing_details' (which might include photos) or 'analyzeListingPhotos'. It doesn't mention prerequisites, constraints, or typical use cases, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.