Server Quality Checklist

Tool Scores

• get_service_request_listB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of behavioral disclosure. It only states it is a bulk lookup, but does not disclose rate limits, authentication requirements, behavior on invalid numbers, or that it is a read operation (implicit but not explicit).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One clear sentence with no wasted words. It is appropriately front-loaded with the core action. Not a 5 because it could be slightly more informative without sacrificing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple bulk lookup with one parameter, the description is adequate but lacks information about return values or error handling, especially given no output schema. Completeness is just satisfactory.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description of the array of strings. The tool description adds no additional meaning beyond the schema; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (bulk look up) and resource (311 service requests) with a specific method (by their numbers in a single call). It distinguishes from sibling tools like get_service_request (single lookup) and get_calendar/get_status (unrelated).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for bulk lookup but provides no explicit guidance on when to use this tool versus alternatives (e.g., get_service_request for single lookups). No exclusions or context provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_calendarA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses timezone interpretation and default behavior, and the 90-day limit, but lacks details on error handling, authentication, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the most critical information (what tool gets, parameters, and important constraints). Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers main functionality, timezone, default, and range limit. Missing output format description and error conditions, but acceptable for a simple calendar tool with no output schema and three well-documented parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds meaning beyond the schema: explains that 'date' can serve as both from and to, and clarifies the range max of 90 days, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Get' and resource 'Alternate Side Parking, Garbage & Recycling, and Schools status' for a date or date range, distinguishing it from sibling tools (get_service_request, get_service_request_list, get_status) which deal with service requests and status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on date range max (90 days) and timezone (America/New_York), but does not explicitly state when to use this tool versus siblings or any exclusions. No when-not or alternative tool guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly indicates a read operation ('Get the current city status') and lists the types, but does not disclose return format or caching behavior. However, it is consistent and straightforward.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action, followed by a concise list of types. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is fairly complete. It explains the action and the available types, though it could mention that the returned data is the current status of that type.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter having enum and description. The description adds semantic value by explaining each enum value in parentheses (e.g., 'Code Blue (cold weather)'), enriching beyond the schema's bare enum list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the current city status' and lists five specific types, making it distinct from sibling tools like get_calendar and get_service_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly guide when to use this tool vs alternatives. It lists the status types but lacks context like 'use this for weather conditions, use get_service_request for specific requests.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_requestA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description bears full responsibility. It discloses the return fields (agency, problem type, status, timestamp, address), indicating it is a read operation. It does not mention error cases or limitations, but for a simple lookup, this is sufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 17 words that immediately states the action and resource, followed by the input format and output fields. Every word contributes useful information; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description is largely complete. It covers what, how, and what to expect. It could mention authentication or pagination but those are unnecessary for a single-lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter is already documented. The description adds value by specifying the exact format ('311-XXXXXXXX'), which is more helpful than just the schema description. This exceeds the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action ('look up one 311 service request') and the resource ('by its number'). It distinguishes from sibling tool 'get_service_request_list' which lists multiple requests. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the input format and what the tool returns. It implicitly differentiates from siblings (single vs list), but does not explicitly state when not to use or provide alternatives. Some guidance is provided through context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.