Server Quality Checklist

Tool Scores

• cancel_orderA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full behavioral disclosure. It indicates a destructive action ('Cancel') but does not clarify if the cancellation is reversible, what happens to associated labels, or whether special permissions are needed. The description adds minimal behavioral context beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste. The first sentence states the core action, and the second provides a critical constraint. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single param, no output schema, no annotations), the description is mostly adequate but lacks any mention of return values, error conditions, or side effects. It does not specify what happens on success or failure, which would help the agent handle responses.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single required parameter 'orderIdentifier', and the schema description is self-explanatory ('The orderIdentifier to cancel'). The description adds no additional parameter meaning, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Cancel', the resource 'Royal Mail Click & Drop order', and the critical precondition 'Must be done before the order is manifested/despatched', making the purpose unambiguous and distinct from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states a timing constraint ('before order is manifested/despatched') and implies this tool is for cancellation only. However, it does not mention what to do if the order is already manifested or suggest alternative tools like track_order for status checking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• track_orderA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool is read-only (Get) and focuses on status/tracking, which is appropriate. However, it does not disclose any behavioral traits like data freshness, rate limits, or potential errors. With no annotations, a 3 is reasonable but could be improved.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no waste. It front-loads the purpose and is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one required parameter and no output schema, the description is largely complete. It explains the tool's purpose and expected input. Minor gap: it could mention that the output contains tracking details, but this is implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter 'orderIdentifier' already described in schema. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get'), clearly identifies the resource ('current status and tracking details'), and specifies the domain ('Royal Mail Click & Drop order'). It distinguishes the tool from siblings like book_order or cancel_order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (after booking an order, to check status/tracking), but does not explicitly state when not to use it or mention alternatives. Since there is no sibling with similar purpose, no explicit exclusion is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_labelA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It reveals that the output is base64-encoded PDF, which is helpful. However, it does not mention any side effects, authentication needs, or whether it is a read-only operation (likely read-only but not explicit).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the core purpose, and includes a key detail about the return format. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description covers the essential purpose and output format. It could mention that the label is for printing or include a link to orderIdentifier documentation, but overall it is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the single parameter with high coverage (100%), and the description mentions it ('orderIdentifier returned when booking the order'). This adds context by linking the parameter to a previous step, which is useful but not transformative given schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('shipping label'), and the context ('Royal Mail Click & Drop order'). It also specifies the return format ('base64-encoded PDF label'), which adds precision. This distinguishes it from siblings like book_order or cancel_order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving a label after booking, but does not explicitly state when to use it versus alternatives. It mentions the input parameter ('orderIdentifier') but does not provide guidance on prerequisites or conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses that the list may vary by account, which is a key behavioral trait (dynamic response based on account), and implies a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose and output, second adds important caveat. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description is sufficient to understand what the tool does and its constraints. Could optionally mention return format (e.g., list of objects) but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no parameters (0 params) and schema description coverage is 100%, so no additional param info needed. Description adds value by stating the output will include Service Register codes and account dependency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists supported Royal Mail and Parcelforce services with Service Register codes, differentiating it from sibling tools like book_order or cancel_order by focusing on service listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions availability depends on Click & Drop account, implying account setup prerequisite, but no explicit when-to-use or comparison with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_orderA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates the tool returns an orderIdentifier, which is useful. However, since no annotations are provided, the description carries full burden for behavioral disclosure. It does not mention mutability, side effects, prerequisites (e.g., account setup), or error conditions. It is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with 15 words, front-loading the key action and outcome. Every word serves a purpose. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (17 parameters, nested objects, no output schema), the description is concise but omits details like what happens on failure, pricing implications, or whether label retrieval is synchronous. However, the schema is well-documented, and the return value is stated. The description is nearly complete for the tool's core function.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, meaning all parameters include descriptions. The tool description itself does not repeat parameter details, but the schema already provides sufficient meaning. However, the description highlights the return value (orderIdentifier), which adds context beyond the schema. Given high schema coverage, baseline is 3, but the explicit mention of the return value and the tool's core action adds value, justifying a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to book a Royal Mail shipment via Click & Drop. It specifies the action (book), resource (shipment), and the system (Click & Drop), and mentions the return value (orderIdentifier). This distinguishes it from siblings like cancel_order, get_label, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives like cancel_order or list_services. It implies usage for booking shipments, but no exclusions or alternatives are mentioned. The context of sibling tools is present, but the description lacks explicit usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.