Server Quality Checklist

Tool Scores

• search_venuesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It discloses some traits (apps defaults to all, limit is 'per provider'), but lacks information on permissions, pagination, rate limits, error handling, or how conflicting address and latitude/longitude inputs are resolved. This is insufficient for a search tool with 8 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the purpose. The parameter list is structured clearly with one line per parameter, each accompanied by a practical example. No redundant content or unnecessary detail is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the search semantics and most parameters, and the output schema handles return values. However, it lacks usage guidelines and omits explicit mention of the latitude/longitude parameters. For a tool with this complexity, the description is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides meaningful semantics for query, address (including 'lat,lon' format), apps, free_delivery, min_rating, and limit with examples. However, latitude and longitude are only implied through 'or provide latitude+longitude' and are not explicitly named, leaving a minor gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Searches for restaurants matching the query' with a concrete example ('pizza'). This distinguishes it from sibling tools like search_items (items) and get_venue_menu (menus), making the resource and verb specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives. There is no mention of using search_items for item-level searches or list_providers for provider discovery. The parameter explanations imply usage but do not state exclusions or preferred contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_venue_menuA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavior on its own. It only says 'Fetches a restaurant menu' without mentioning return format, error cases, or how optional parameters behave. Given the presence of an output schema, some detail is expected, but the limited description leaves most behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with a clear purpose statement followed by a parameter list. The phrasing around address is slightly awkward, but every sentence contributes useful information without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the five parameters and no schema descriptions, the tool description covers the core inputs and links to search_venues, but it omits details about output format, optional parameter behavior, and how latitude/longitude should be provided together. This is adequate for a simple fetch tool but leaves some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value by specifying allowed provider values ('wolt' or 'glovo'), the source of venue_id ('from search_venues results'), and the flexible address format ('lat,lon' or 'latitude+longitude'). However, it does not fully clarify the separate latitude and longitude parameters or distinguish required vs optional fields, so it only partially compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetches a restaurant menu' with a specific verb and resource. It distinguishes from siblings like search_venues and search_items by focusing on the menu retrieval action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context for usage by noting that venue_id comes from 'search_venues results', implying the correct sequence. It does not explicitly exclude alternatives, but the guidance is clear enough for a simple retrieval tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_itemsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of disclosing behavior. It does not state whether the operation is read-only, whether authentication is needed, or what happens if no results match. It only explains parameters, not the tool's general behavior or return conventions. This is a significant gap for a tool that likely performs external queries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of one precise sentence followed by a clear, well-formatted parameter list. Every line adds value by explaining a parameter, with no redundant fluff. It is both concise and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 10 parameters and an output schema, the description covers the input semantics reasonably well but lacks broader contextual details such as typical use cases, filtering behavior, pagination, or any caveats about location-based searches. It is adequate for a straightforward search tool but does not fully orient an agent on how to interpret results or handle edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description is the only source of parameter meaning. It adds useful details: examples for max_price_cents (5000 = 50 PLN), a scale for min_rating (0-10), explanation of free_delivery (True = restaurants with free delivery only), and the list of sort options. It also mentions latitude/longitude as an alternative to an address. However, it does not elaborate on the 'apps' parameter beyond saying 'list of apps,' and latitude/longitude are not listed as distinct named parameters, though they are referenced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Searches for dishes matching the query.' This clearly distinguishes it from sibling tools like search_venues (which searches venues) and get_venue_menu (which retrieves a menu). The example 'pizza margherita' further reinforces the intended use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for searching dishes, which gives clear context for when to use it, but it does not explicitly state when not to use it or mention alternatives. No exclusions or comparisons to siblings are provided, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_locationA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that input is resolved to coordinates, but omits potential failure modes, output format details, whether network calls are involved, or any limitations. This is minimal disclosure beyond the core function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the action and provides examples without any extraneous wording. It is concise and immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool with an output schema present, the description covers the essential purpose and input flexibility. It lacks explicit usage guidelines and behavioral details, but these are secondary given the tool's simplicity and the presence of an output schema. Overall, it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema simply defines 'address' as a string with no description. The tool description adds substantial meaning by clearly stating the parameter can be either a human-readable address or a 'lat,lon' pair, with a concrete example. This fully compensates for the schema's lack of parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb 'Resolves' with a clear resource ('an address ... or 'lat,lon'') and the outcome ('to coordinates'). Concrete examples ('Warszawa, Marszałkowska 98') and the explicit 'lat,lon' format make the purpose unmistakable and distinguish it from sibling tools that list providers, search venues, search items, or get venue menus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage scenario by showing acceptable input formats, but it does not explicitly state when to use this tool versus alternatives or mention exclusions. For example, it does not say whether to use this before search_venues or if you already have coordinates to use elsewhere. This is adequate but lacks clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_providersA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It states 'Returns', which signals a read-only operation with no side effects, but it does not disclose whether authentication is needed, ordering, or other behavioral aspects. The simplicity of the tool makes this acceptable but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler, front-loading the action and resource. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is intentionally simple with no parameters and an output schema present. The description fully covers the tool's purpose, and additional behavioral details are unnecessary given the output schema handles return value specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are not relevant. Per the rubric, a baseline of 4 is appropriate for 0-parameter tools. The description does not need to explain any input details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Returns' and identifies the resource as 'list of available food delivery apps'. This clearly distinguishes it from sibling tools like search_venues and get_venue_menu, which focus on searching or menus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool returns available food delivery apps, but it does not explicitly state when to use it or mention alternatives. Since the tool has zero parameters, the context is clear enough to infer use before searching for venues or items, but explicit guidance is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.