Server Quality Checklist

Tool Scores

• add_addressC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states this is an 'Add' operation, implying a write/mutation, but doesn't disclose behavioral traits like required permissions, whether the address becomes immediately available, potential rate limits, error conditions, or what happens on success/failure. The description is minimal and lacks operational context needed for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste, front-loading the core purpose. It could be slightly more structured by separating usage context, but it efficiently communicates the essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 13 parameters, no annotations, and no output schema, the description is incomplete. It lacks information on authentication needs (implied by 'integration_key' but not explained), response format, error handling, or side effects. The context signals indicate high complexity, but the description doesn't compensate for missing structured data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 13 parameters. The description adds no parameter-specific semantics beyond implying 'type' can be 'shipper' or 'receiver' (which is already in the schema). It doesn't clarify relationships between parameters or provide examples. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add a new address') and resource ('address book'), with specific purpose for 'saving shipper or receiver addresses for reuse'. It doesn't explicitly differentiate from sibling tools like 'edit_address' or 'delete_address', but the verb 'Add' implies creation rather than modification or removal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the tool is 'Used for saving shipper or receiver addresses for reuse', which provides some context about when addresses might be added. However, it offers no guidance on when to use this versus alternatives like 'edit_address' for existing addresses, or prerequisites such as authentication via 'integration_key'. No explicit when-not-to-use or sibling comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_addressC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an update operation, implying mutation, but doesn't cover permissions, error conditions, side effects, or response format. The partial update hint ('Only changed fields need to be provided') adds some value but leaves critical behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences with zero wasted words. It front-loads the core purpose and follows with a useful behavioral hint, making it efficient and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 13 parameters, no annotations, and no output schema, the description is insufficient. It lacks details on permissions, error handling, return values, and how it differs from sibling tools. The partial update hint is helpful but doesn't compensate for the overall gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 13 parameters. The description adds minimal value beyond the schema by hinting at partial updates ('Only changed fields need to be provided'), but doesn't elaborate on parameter interactions or semantics not already in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update') and resource ('an existing address in the address book'), making the purpose unambiguous. However, it doesn't explicitly differentiate from its sibling 'add_address' beyond the 'existing' qualifier, which is implied but not directly contrasted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'add_address' or 'delete_address'. It mentions 'Only changed fields need to be provided,' which is a usage hint but not a contextual guideline for tool selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_addressC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves an address by ID, implying a read-only operation, but doesn't cover aspects like error handling, authentication needs, rate limits, or response format. For a tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., address details), error conditions, or dependencies like authentication. For a read operation with no structured output information, the description should provide more context to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for both parameters ('integration_key' and 'id'). The description adds no additional parameter semantics beyond what the schema provides, such as format details or usage examples. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a specific address by ID from the address book.' It specifies the verb ('Get'), resource ('address'), and scope ('by ID from the address book'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'list_addresses' or 'get_account_info', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_addresses' for multiple addresses or 'get_account_info' for other data, nor does it specify prerequisites or exclusions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_shipping_statsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'gets' and 'views' data, implying a read-only operation, but fails to mention critical behaviors such as authentication needs (implied by integration_key), rate limits, response format, or whether it aggregates real-time vs. historical data. This leaves significant gaps for a tool with four parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, using a single sentence that efficiently lists key data points without redundancy. Every phrase ('shipping statistics and analytics', 'view shipment counts...') directly supports the tool's purpose, with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, no output schema), the description is incomplete. It lacks details on authentication, response structure, error handling, and how parameters interact (e.g., period vs. custom dates). While concise, it does not adequately compensate for the missing structured data, leaving the agent under-informed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all four parameters (integration_key, period, date_from, date_to). The description adds no parameter-specific semantics beyond implying a temporal focus through terms like 'daily trends', which aligns with the period/date parameters but does not provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('get', 'view') and resources ('shipping statistics and analytics'), listing concrete data points like shipment counts, cost breakdowns, and carrier usage. However, it does not explicitly differentiate this tool from potential sibling analytics tools (e.g., get_account_info might also provide stats), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., requiring an integration_key), exclusions, or comparisons to sibling tools like get_account_info or search_shipments, leaving the agent to infer usage context solely from the tool name and parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_shipmentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions pagination and filters but fails to describe critical behaviors such as response format, error handling, rate limits, authentication requirements (though 'integration_key' hints at it), or whether this is a read-only operation. This leaves significant gaps for a tool with 7 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core functionality (list shipments with pagination and filters) and specifies key filter options. There is no wasted verbiage, and every word contributes to understanding the tool's scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, no output schema, no annotations), the description is incomplete. It lacks details on behavioral traits, response structure, error cases, and usage context relative to siblings. For a list operation with multiple filters and pagination, more guidance is needed to help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema fully documents all 7 parameters. The description adds minimal value beyond the schema by listing filter types (status, carrier, date range) and mentioning pagination, but it doesn't provide additional context like parameter interactions or default behaviors. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as listing shipments with pagination and filters, specifying the resource (shipments) and key capabilities. It distinguishes from siblings like 'get_shipment' (singular) and 'search_shipments' by emphasizing pagination and specific filters, though it doesn't explicitly contrast with 'search_shipments'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_shipments' or 'get_shipment'. It mentions filtering capabilities but doesn't specify scenarios, prerequisites, or exclusions, leaving the agent to infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_shipmentsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the search functionality and return of matching shipments but lacks details on permissions, rate limits, error handling, or pagination behavior. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded in a single sentence, efficiently stating the tool's purpose without unnecessary details. However, it could be slightly improved by integrating usage guidelines or behavioral context to enhance clarity without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It covers the basic purpose but fails to address behavioral aspects like authentication needs (implied by 'integration_key'), result limitations, or error scenarios. For a search tool with multiple parameters and no structured output, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents the parameters (integration_key, q, limit). The description mentions 'order ID or tracking number' for the 'q' parameter, adding minimal context but not significant value beyond the schema. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search shipments by order ID or tracking number. Returns matching shipments.' It specifies the verb ('search'), resource ('shipments'), and search criteria. However, it does not explicitly differentiate from sibling tools like 'list_shipments' or 'get_shipment', which might offer different scopes or filtering capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools such as 'list_shipments' (which might list all shipments without search) or 'get_shipment' (which might retrieve a single shipment by ID), leaving the agent to infer usage from context without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states it's a read operation ('Get'), implying safety, but doesn't cover permissions, rate limits, error handling, or response format. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and lists key data types without unnecessary words. Every part earns its place by clarifying scope, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it lacks details on behavioral traits and return values. It covers the 'what' but not the 'how' or 'what happens next,' leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'integration_key' documented in the schema. The description doesn't add any parameter-specific details beyond what the schema provides, such as explaining why this key is needed or its format. Baseline 3 is appropriate since the schema handles the parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('Shipi account information'), listing the types of information retrieved (user details, store info, billing/balance, plan, feature flags). It distinguishes itself from siblings by focusing on account-level data rather than shipments, addresses, or carriers, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an integration key), context for retrieval, or exclusions. Sibling tools like 'get_shipment' or 'get_address' handle specific resources, but no comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_carrierB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns 'carrier type and shipper address info,' which hints at read-only behavior, but doesn't confirm if it's safe (e.g., non-destructive), mention authentication needs (like the 'integration_key' parameter), rate limits, or error handling. This is inadequate for a tool with parameters and no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it states the core purpose in the first clause and adds return details in the second. Both sentences earn their place by providing essential information without redundancy or fluff, making it efficient for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally complete. It covers the basic purpose and return values, but lacks details on usage context, behavioral traits, or error handling. With no output schema, it should ideally explain return structure more, but the mention of 'carrier type and shipper address info' provides some guidance, keeping it adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('integration_key' and 'id') with descriptions. The description adds minimal value by implying the 'id' parameter is used to fetch details, but doesn't provide additional semantics like format examples or usage context beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get details of a specific carrier account by ID.' It specifies the verb ('Get details'), resource ('carrier account'), and key identifier ('by ID'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'list_carriers' or 'get_account_info,' which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_carriers' (for listing all carriers) or 'get_account_info' (which might overlap in functionality), nor does it specify prerequisites or exclusions. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_addressB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the tool deletes an address, implying a destructive mutation, but fails to disclose critical behavioral traits like whether deletion is permanent, requires specific permissions, has side effects on related shipments, or provides confirmation feedback. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loading the core action ('Delete an address') and purpose ('by ID'). Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral implications (e.g., permanence, permissions), error handling, or what happens post-deletion. For a tool that modifies data, this leaves the agent under-informed about critical usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters ('integration_key' and 'id'). The description adds no additional meaning beyond implying 'id' is used for deletion, which is already clear from the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete') and resource ('an address from the address book'), and distinguishes it from siblings like 'edit_address' or 'get_address' by specifying the destructive nature of the operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'edit_address' or 'list_addresses', nor does it mention prerequisites such as needing a valid address ID or integration key. The description only states what it does, not when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_shipmentB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the tool as a read operation ('Get detailed information'), which is clear, but fails to mention critical aspects such as authentication needs (implied by 'integration_key' parameter), rate limits, error handling, or whether it requires specific permissions. The description adds minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured in a single sentence, front-loaded with the core purpose and followed by specifics on included data fields. Every word earns its place, with no redundancy or unnecessary details, making it efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose and data fields but lacks details on authentication, error handling, or output structure. Without annotations or an output schema, the description should provide more context to fully guide usage, but it meets the minimum viable threshold.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all three parameters. The description adds marginal value by clarifying that the tool can use 'ID or order ID' to retrieve shipment details, but does not provide additional semantics beyond what the schema specifies (e.g., how 'integration_key' relates to authentication or the format of IDs). Baseline 3 is appropriate as the schema handles most parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed information about a specific shipment by ID or order ID.' It specifies the verb ('Get'), resource ('shipment'), and scope ('detailed information'), but does not explicitly differentiate it from sibling tools like 'list_shipments' or 'track_shipment' beyond mentioning the included data fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it retrieves information 'by ID or order ID,' suggesting it's for specific shipments rather than listing all. However, it lacks explicit guidance on when to use this tool versus alternatives like 'list_shipments' (for bulk retrieval) or 'track_shipment' (for tracking updates), and does not mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_addressesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a list operation (implying read-only) and mentions filtering, but doesn't cover critical aspects like pagination, rate limits, authentication needs (though 'integration_key' param hints at this), error conditions, or what the output looks like. For a tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - two clear sentences with zero wasted words. The first sentence establishes the core purpose, the second adds the key capability. It's front-loaded with the essential information and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (list operation with filtering), no annotations, no output schema, and 100% schema coverage, the description is minimally adequate. It covers what the tool does but lacks information about behavioral characteristics, output format, and usage context that would be helpful for an AI agent. The high schema coverage helps, but the absence of output information and behavioral context keeps this from being complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description adds marginal value by mentioning the filtering capability ('Filter by type') which aligns with the 'type' parameter, but doesn't provide additional semantic context beyond what's in the schema descriptions. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'all saved addresses from the address book', making the purpose immediately understandable. It distinguishes this from other address-related tools like 'get_address' (singular) and 'add_address'/'edit_address'/'delete_address' (mutations). However, it doesn't explicitly differentiate from 'search_shipments' or 'list_shipments' which might also involve addresses, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving addresses with optional filtering by type, but provides no explicit guidance on when to use this versus alternatives like 'get_address' (for a specific address) or 'search_shipments' (which might include address data). It mentions filtering capability but doesn't specify prerequisites or when-not-to-use scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_pickupB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the action ('Schedule a carrier pickup') and outcome ('carrier will come...'), but lacks critical behavioral details: whether this is a mutating operation (likely yes), permission requirements, rate limits, error conditions, or what happens if pickup fails. For a tool with no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The first sentence states the core purpose, and the second adds useful context about the carrier's action. It's front-loaded and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is minimal but covers the basic purpose. However, for a mutating tool with 5 parameters, it should ideally include more about behavioral traits, error handling, or output expectations. It's adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no parameter-specific information beyond what's in the schema (e.g., it doesn't clarify 'carrier_type' options or 'pickup_time' format). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Schedule a carrier pickup') and resource ('for a shipment'), with additional context about what the carrier will do ('come to the shipper address to collect the package'). It distinguishes itself from siblings like 'create_shipment' or 'track_shipment' by focusing on pickup scheduling rather than creation or tracking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a shipment created first), exclusions (e.g., not for already picked-up shipments), or comparisons to sibling tools like 'cancel_shipment' or 'get_shipping_rates'. Usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_shipmentB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Cancel a shipment and void its label'), which implies a destructive mutation, but does not disclose critical traits such as permission requirements, whether the cancellation is reversible, rate limits, or error conditions. This leaves significant gaps for an agent to understand the tool's behavior fully.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and includes essential parameter information. There is no wasted text, and it directly communicates the tool's purpose and requirement without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a destructive mutation tool with no annotations and no output schema, the description is insufficient. It lacks details on behavioral traits (e.g., permissions, reversibility), expected outcomes, error handling, and how it differs from siblings. For a tool that performs a significant action like cancellation, more context is needed to guide an agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('integration_key' and 'shipment_id'). The description adds minimal value by mentioning 'shipment_id' as required, but does not provide additional semantics beyond what the schema specifies, such as format details or usage context for 'integration_key'. Baseline 3 is appropriate given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Cancel a shipment and void its label') and identifies the resource ('shipment'), distinguishing it from siblings like 'get_shipment' or 'track_shipment'. It explicitly mentions the outcome of voiding the label, which adds specificity beyond just cancellation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a shipment needs to be canceled and its label voided, but it does not explicitly state when to use this tool versus alternatives (e.g., no comparison to editing or other modifications). It mentions the required parameter 'shipment_id', which provides some context for prerequisites, but lacks explicit guidance on scenarios or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_shipping_ratesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'live shipping rates' and 'compare prices,' which suggests a read-only operation, but fails to detail critical aspects like authentication needs (e.g., integration_key usage), rate limits, error handling, or whether this is a simulation or actual booking step, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two concise sentences that directly state the tool's function and required inputs without any redundant or extraneous information, making it efficient and easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters with nested objects, no output schema, and no annotations), the description is adequate but incomplete. It covers the basic purpose and inputs but lacks details on output format, error cases, or behavioral nuances, which are crucial for an agent to invoke it correctly in a shipping context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by mentioning 'recipient address and package details,' which aligns with the schema but does not provide additional syntax, format, or usage details beyond what the schema specifies, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get live shipping rates') and resources ('from all configured carriers'), and distinguishes it from siblings by specifying its unique function of comparing prices across multiple carriers (FedEx, UPS, DHL, USPS, etc.), unlike tools like 'get_carrier' or 'get_shipment'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by stating 'Provide recipient address and package details to compare prices,' but it lacks explicit guidance on when to use this tool versus alternatives like 'get_shipment' or 'track_shipment,' and does not mention prerequisites or exclusions, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_carriersA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that credentials are never exposed, which is a useful security detail, but does not mention other behavioral traits like whether the operation is read-only, potential rate limits, error conditions, or response format. The added context is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with zero waste: the first explains the purpose and output, and the second adds a security note. It is front-loaded with the core functionality, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema), the description is adequate but has gaps. It covers what the tool does and a security aspect, but lacks usage guidelines, behavioral details like read-only status, and output information. For a simple list tool, this is minimally viable but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'integration_key'. The description adds no additional meaning about parameters, such as when to use the integration_key or its impact on results. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all configured shipping carrier accounts'), specifying what information is shown ('carrier type, primary status, and shipper address'). It distinguishes from siblings like 'get_carrier' by indicating it lists all carriers rather than retrieving a specific one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'get_carrier' for a single carrier or 'get_account_info' for broader account details. It lacks context about prerequisites or typical use cases, offering only basic functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_labelsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns label URLs and supports filtering by printed status, which are useful behavioral traits. However, it doesn't mention pagination behavior (implied by page/limit parameters), rate limits, authentication needs, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The first states the core purpose, the second adds filtering and return value information. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with 4 parameters and no output schema, the description provides adequate context about what it fetches and returns. However, without annotations or output schema, it lacks details about response format, error handling, and authentication requirements that would be helpful for complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description adds marginal value by mentioning filtering by printed/unprinted status, which aligns with the 'printed' parameter. No additional parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('fetch') and resource ('shipping labels'), specifies the purpose ('for printing'), and distinguishes it from siblings by focusing on label retrieval rather than shipment/address operations. It's specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when labels need to be printed and filtered by status, but doesn't explicitly state when to use this tool versus alternatives like 'get_shipment' or 'list_shipments' that might provide related data. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• track_shipmentA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions auto-detection of carrier, which is a useful behavioral trait, but lacks details on permissions, rate limits, error handling, or what the tracking URL output entails. For a tool with no annotations, this leaves significant gaps in understanding its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of two concise sentences that directly state the tool's function and a key feature (auto-detection). Every sentence earns its place without redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely returns a tracking URL. It adequately covers the basic purpose but lacks details on output format, error cases, or integration context, which are important for full understanding in this shipping domain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds marginal value by explaining that carrier auto-detection occurs if omitted, but does not provide additional semantics beyond what the schema specifies, such as format examples or carrier code details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('tracking URL for a shipment'), and distinguishes it from siblings by focusing on tracking rather than creation, cancellation, or listing of shipments. It adds unique value by mentioning auto-detection of carrier from tracking number format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool—to obtain a tracking URL—and implies usage by mentioning auto-detection if carrier is omitted. However, it does not explicitly state when not to use it or name alternatives among siblings, such as 'get_shipment' which might provide more detailed shipment info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_shipmentA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that this is a creation/mutation operation (implied by 'Create'), mentions required inputs, and describes the return values (tracking number and label URL). However, it lacks details on permissions, rate limits, error conditions, or whether this action is reversible (e.g., via 'cancel_shipment').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences that front-load the core action, list prerequisites, and state the return values. Every word earns its place with no redundancy or fluff, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations and no output schema, the description is adequate but has gaps. It covers the purpose, prerequisites, and return values, which is sufficient for basic use. However, it lacks details on behavioral aspects like error handling, side effects, or integration with sibling tools (e.g., 'cancel_shipment'), leaving room for improvement given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by mentioning the general categories of inputs (carrier account, addresses, product details) but doesn't provide additional syntax, format, or constraint details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a shipping label') and distinguishes it from sibling tools like 'fetch_labels', 'get_shipment', or 'track_shipment'. It specifies the exact resource being created and the outcome, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states prerequisites ('Requires carrier account, shipper/recipient addresses, and product details'), which provides clear context for when to use this tool. However, it doesn't mention when not to use it or name specific alternatives among siblings (e.g., 'get_shipping_rates' for rate checking before creation).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.