Server Quality Checklist

Tool Scores

• search_servicesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool returns 'a list of matching services with basic information,' which gives some behavioral context about the output format. However, it lacks details on permissions, rate limits, pagination, error handling, or what constitutes 'basic information.' For a search tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in two sentences: one stating the action and scope, and another describing the output. Every sentence adds value without redundancy, making it easy to parse and front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a search function with one parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't explain the return format beyond 'basic information,' error cases, or how results are ordered/filtered. For a tool with zero structured coverage, more contextual details are needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, with one parameter 'query' undocumented in the schema. The description adds minimal semantics by implying 'query' is used to search 'by name or description,' but it doesn't specify format, constraints, or examples. This partially compensates for the coverage gap but falls short of fully explaining the parameter's purpose and usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for Berlin administrative services by name or description.' It specifies the verb (search), resource (Berlin administrative services), and scope (by name or description). However, it doesn't explicitly differentiate from sibling tools like 'list_services' or 'get_service_details,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions returning 'basic information' but doesn't clarify when to choose this over 'get_service_details' for more details or 'list_services' for unfiltered listings. No explicit when/when-not instructions or prerequisites are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_detailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool returns 'comprehensive information including requirements, forms, fees, appointments, and more,' which gives some context on output content. However, it lacks details on error handling (e.g., invalid ID), rate limits, authentication needs, or whether it's a read-only operation. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, and the second adds valuable context about return content. Both sentences earn their place by providing essential information without redundancy or fluff. It's efficient and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 1 parameter) and lack of annotations and output schema, the description is minimally complete. It covers the purpose and output scope but misses behavioral details like error cases or authentication. It's adequate for basic use but has clear gaps that could hinder an agent in edge cases or integration scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It adds meaning by specifying that 'service_id' refers to 'a specific Berlin service,' clarifying the parameter's purpose beyond the schema. However, it doesn't detail the ID format (e.g., numeric, string pattern) or provide examples. Since there's only one parameter, the description does a good job of contextualizing it, earning a baseline 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed information about a specific Berlin service by its ID.' It specifies the verb ('Get'), resource ('Berlin service'), and key constraint ('by its ID'). However, it doesn't explicitly differentiate from sibling tools like 'get_services_stats' or 'list_services' beyond mentioning 'detailed information' versus potentially summary views.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_services' (for browsing) or 'search_services' (for filtering), nor does it specify prerequisites such as needing a valid service ID. Usage is implied by the need for detailed info on a known service, but no explicit context or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination and return format (names and IDs), which is helpful, but lacks details on rate limits, authentication needs, error handling, or whether it's read-only. The description doesn't contradict annotations, but it's insufficient for a mutation-free tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and efficiently adds details about pagination and return format. Every sentence contributes value, with no wasted words, though it could be slightly more structured for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter, no output schema, no annotations), the description covers the basic purpose and return format adequately. However, it misses parameter documentation and lacks behavioral context like pagination mechanics or error scenarios, making it incomplete for optimal agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It doesn't mention the 'limit' parameter at all, leaving it undocumented. Since there's only one parameter, the baseline is higher, but the description adds no parameter-specific information beyond the general context of listing services.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('Berlin administrative services'), specifying it returns a paginated list with names and IDs. However, it doesn't explicitly differentiate from sibling tools like 'search_services' or 'get_services_stats' beyond mentioning it lists 'all available' services.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it lists 'all available' services, suggesting it's for broad retrieval rather than filtering or detailed analysis. However, it doesn't provide explicit guidance on when to use this versus alternatives like 'search_services' for filtered results or 'get_service_details' for specifics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_services_statsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool retrieves statistics (implying a read-only operation) and hints at the type of data returned, but does not specify behavioral traits such as rate limits, authentication needs, or whether the data is real-time or cached. The description is consistent with a read operation but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get statistics about the Berlin services dataset') and adds clarifying examples ('total count, last update, etc.') without unnecessary elaboration. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is adequate but minimal. It covers the basic purpose and hints at return values, but does not fully explain the output format or provide rich context for a statistical tool, leaving gaps in completeness for an agent to understand the full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not add parameter details beyond the schema, which is appropriate here. A baseline score of 4 is applied for tools with no parameters, as there is no gap to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('statistics about the Berlin services dataset'), including examples of what statistics are returned ('total count, last update, etc.'). It distinguishes from siblings by focusing on aggregated statistics rather than individual details, lists, or searches, though it doesn't explicitly name the sibling tools for comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'statistics about the Berlin services dataset,' suggesting it should be used for aggregated metrics rather than detailed or filtered data. However, it lacks explicit guidance on when to use this tool versus alternatives like 'list_services' or 'search_ervices,' and does not mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.