Server Quality Checklist

Tool Scores

• list_great_roastersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions listing/filtering, implying a read-only operation, but does not disclose return format, ordering, or the exact meaning of 'curated.' Given the absence of annotations, more behavioral context would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action ('List...'), and each sentence adds value: the first defines the tool, the second provides use cases. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with 3 optional parameters and full schema coverage, the description gives adequate purpose and use-case context. However, since there is no output schema and no annotations, the agent might not know what fields are returned. Still, the description covers the core invocation context well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions, so the baseline is 3. The description only restates filtering by country and reputation tier, adding no extra semantic detail beyond the schema. The 'limit' parameter is not mentioned in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'List curated specialty coffee roasters from the database, filtered by country and reputation tier.' This distinguishes it from sibling tools like search_cafes or get_cafe_details, which focus on different entities (cafes, details).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear use cases: 'finding cafes that serve a given roaster's beans, or planning a roaster-focused trip.' This gives contextual guidance on when to use the tool, though it does not explicitly mention alternatives or exclusions relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_cafesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It adds useful context: the database is 'curated', results are 'scored', and sorted by espresso-quality score descending. It doesn't mention pagination or result format, but for a search tool this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and contains no filler. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately complex search tool with 6 parameters but no output schema or annotations, the description plus full schema descriptions provide a complete picture: what is searched, the filter fields, and the result ordering. It lacks explicit mention of pagination/limit, but the schema's default handles that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description repeats some parameter names (city, country, roaster, min_score) but adds no additional semantics beyond the schema. It also says 'by name' though there is no explicit 'name' parameter, potentially confusing the query parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a curated specialty coffee cafe database by multiple filters (city, country, roaster, min score) and returns scored results. This distinguishes it from sibling tools like get_cafe_details (specific details) and find_espresso_near (location-based).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by listing searchable fields and result ordering, but it does not explicitly state when to prefer this tool over alternatives or mention any exclusions. For example, it doesn't say 'for a specific cafe's details, use get_cafe_details.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_espresso_nearA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It reveals that results are sorted by espresso-quality score and include distance and reasoning, giving insight into output behavior. It does not mention rate limits or error handling, but for a read-only search tool, these are secondary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences deliver all key information: action, resource, filter criteria, ranking, and return details. No filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the core return aspects (sorted results, distance, score reasoning) but lacks a detailed output schema. It doesn't specify sorting direction (ascending/descending) or the exact response shape, but given the schema's thorough parameter coverage, this is adequate for a simple search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameters with descriptive comments, so the description does not need to re-explain them. The description adds a slight behavioral context by mentioning coordinates and radius, but it doesn't go beyond the schema in explaining parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: finding ranked specialty espresso cafes near given coordinates. It distinguishes itself from siblings like search_cafes by specifying location-based search, sorting by espresso-quality score, and returning distance and score reasoning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the primary use case: searching for cafes by proximity to specific coordinates. It does not explicitly mention when not to use this tool or suggest alternatives, but the context is clear enough for an agent to choose it over general search tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cafe_detailsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. 'Retrieve' indicates a read-only operation, and the description goes beyond the schema by specifying the output includes a score breakdown and nearby/related cafes. It does not detail error behavior or caching, but for a simple get-by-id tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the main action first, then specifies included content. Every part adds value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), and the description adequately summarizes the response content (record, score breakdown, nearby cafés). However, it doesn't mention if the record is filtered or any limitations, though for a get-by-id this is not a major gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the single 'id' parameter already has a rich description with an example ('tim-wendelboe-oslo') and a pointer to search_cafes. The tool description adds no additional parameter semantics, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve a full curated record for a cafe by id') with a specific resource and scope. It distinguishes itself from siblings like search_cafes (search vs. retrieval by id) and score_cafe (retrieval vs. scoring), while enumerating key content (espresso-quality score breakdown, nearby/related cafes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when you have a cafe id and need detailed information. The schema parameter description adds 'Use search_cafes to find ids,' giving a clear workflow. However, it does not explicitly discuss exclusions or compare against score_cafe or find_espresso_near, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_cafeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the operation (scoring algorithm), the output structure (score, tier, per-signal contributions, reasoning), and that no database lookup is required. It does not detail side effects or error behavior, but as a pure computation tool, these are not likely relevant. The description provides more than minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences. The first sentence states the primary action and key constraint, the second states outputs and usage context. Every sentence is informative, no redundant or filler content. It is well structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (nested observed_signals object, multiple signal types, no output schema), the description covers the essential points: what it does, what it returns, and when to use it. The schema handles parameter details. The only minor omission is not explicitly stating that unknown signals are skipped, but this is covered in the schema. Overall, the description is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description only refers generically to 'observed signals' and adds no parameter-specific meaning beyond what the schema provides. It does not mislead, but it also does not enhance understanding of the parameters. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Apply the espresso-quality scoring algorithm to a set of observed signals.' It also clarifies there is no database lookup and lists concrete outputs. This clearly distinguishes it from sibling tools like get_cafe_details (which would involve lookup) and search_cafes (which finds cafes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context: 'Use this when you've gathered information about a cafe from a website, photo, or review and want a structured assessment.' It also adds 'no database lookup required,' implying a contrast with database-backed tools, though it does not explicitly name alternatives or exclusion criteria. This is clear but lacks explicit 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_anti_patternsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While no annotations are provided, the description discloses that entries 'show why it's flagged,' revealing output structure. It also details the inclusion criteria (mass-market chains, flavor-led specialty, etc.), giving the agent understanding of what to expect. This is meaningful behavioral context beyond a simple list description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with the primary action and purpose in the first sentence. It efficiently packs relevant detail (inclusion criteria, output note, use cases) without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  As a simple list tool with two optional parameters and no output schema, the description provides sufficient information: it explains the tool's scope, the nature of list items (with reasons), and practical use cases. Combined with the schema's parameter coverage, this is complete for the task.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters fully, but the description adds semantic meaning by defining the 'flavor-led specialty' category in prose ('display third-wave signage but lean heavily on flavored drinks') and providing examples like Starbucks, Dunkin', and Costa for mass-market chains. This enriches the enum values beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and defines the resource as anti-pattern coffee shops, explicitly stating they 'exemplify what to AVOID when looking for great espresso.' It further distinguishes from siblings by mentioning mass-market chains and flavor-led specialty shops, clearly contrasting with list_great_roasters. This provides a clear, unambiguous purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states two explicit use cases: 'Useful as contrast when recommending real specialty cafes' and 'as regression fixtures for the scoring algorithm.' It gives clear context for when to leverage this tool, although it does not name alternative tools or explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.