Server Quality Checklist

Tool Scores

• get_ratesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that it gets rates, without mentioning return format, limitations (e.g., US-only), or that it requires ZIP codes rather than full addresses. The mismatch between 'addresses' and the actual ZIP code parameters further reduces transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that directly states the tool's purpose. It is front-loaded and contains no wasteful words, though it is minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has six parameters and no output schema or annotations, the description is too sparse. It does not describe the return value structure, any constraints, or clarify the actual input format (ZIP codes vs addresses), leaving significant gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All six parameters have descriptions in the schema (100% coverage), so the description itself does not need to explain them. The description adds no parameter-level information beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and clearly identifies the resource (UPS shipping rates) and scope (between two addresses for a package). This makes it distinct from sibling tools like track_package and find_locations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. Usage is only implied by the tool's name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_pickupB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure. It only says 'Schedule a UPS package pickup' without explaining what the user can expect after invocation—confirmation, immediate effect, required account, or potential charges. This is a significant gap for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that states the core purpose without any unnecessary words. It is front-loaded and to the point, with no wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a scheduling tool with no annotations, no output schema, and 7 parameters, the description is minimally viable. It conveys the action but lacks context about the result of scheduling, any side effects, or how the requested pickup will be confirmed. The schema covers parameters, but the description does not add enough behavioral or sequential context for an agent to fully understand the invocation expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all 7 parameters, so the schema already documents each field. The tool description adds no parameter-specific meaning beyond what the schema provides, matching the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Schedule') and the resource ('UPS package pickup'), which is distinct from the sibling tools (track_package, get_rates, find_locations). The mention of 'at your address' adds a specific scope that matches the address parameter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as track_package or get_rates. There are no exclusions, prerequisites, or hints about typical use cases beyond the obvious action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_locationsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral transparency. It only states 'Find' which implies a read operation, but it does not disclose the return format, potential filtering behavior, pagination, or any other operational characteristics. Lacks sufficient detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the verb and resource, with no redundant or filler content. Every word contributes to the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the schema covers all parameters, but the description lacks details about the response structure or any limitations (e.g., what the returned locations include, default radius). Given no annotations and no output schema, the description is minimally adequate but could provide more context for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add extra meaning beyond the schema; 'near an address' matches the address parameter but does not elaborate on radius_miles or location_type. The schema already handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Find' and the resource 'UPS Store locations, drop boxes, and access points' near an address. This is specific and distinguishes from sibling tools like track_package, get_rates, and schedule_pickup, which handle different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: when searching for UPS locations near an address. It does not explicitly mention alternatives or exclusions, but the sibling set makes it obvious this is the location lookup tool, so usage is easy to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• track_packageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the operation is a read-only tracking action returning status, history, and delivery estimate, but does not mention potential errors, rate limits, or permissions. It is adequate but lacks deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that conveys purpose and output without extraneous detail. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is sufficient: it specifies the input and expected outputs (status, location history, delivery estimate). It could elaborate on return formats or error handling, but this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter, which already documents the format and example. The description adds no new meaning beyond restating 'by tracking number', so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Track') and resource ('UPS package') with the scope ('by tracking number'). It clearly differentiates from siblings like get_rates and schedule_pickup, which handle different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you need package status, but does not explicitly state when to use this over alternatives or any exclusions. For a simple, well-differentiated tool, context is implied rather than explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.