Server Quality Checklist

Tool Scores

• list_carriersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states the tool lists carriers, but does not mention ordering, filtering, side effects, or scope. Since the tool has no parameters and a likely output schema, the transparency is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no unnecessary words. Every part contributes meaning, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema is provided, the description sufficiently explains the tool's purpose. It could mention that it returns all carriers without filtering, but the context is already complete for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters with 100% coverage. The description adds no parameter info because none is needed; it implicitly indicates no parameters are required. This is a baseline 4 for no-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: to list shipping carriers that the server can create labels for. It uses a specific verb ('List') and resource ('carriers'), and distinguishes itself from sibling tools (create_label, describe_carrier) by its focus on enumeration rather than creation or details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives like describe_carrier or create_label. Usage is implied by the tool's name and purpose, but no direct guidance is given, which leaves the agent to infer from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_labelA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses behavioral traits: input structure, override logic, optional output suppression, and return format. It mentions that config is for testing, implying normal credential loading from environment. It does not mention costs or destructive nature, but the behavior is well-explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: purpose first, then request details, then optional parameters and return value. It is somewhat verbose but every sentence adds value. Front-loading works well.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 params, nested objects, output schema exists), the description covers input structure, overrides, credential handling, and return format. It does not detail all sub-fields but provides sufficient context for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by detailing the nested request structure, address fields, carrier/product, parcels, references, label format, and the purpose of carrier, config, and include_label. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a shipping label from a canonical shipment request,' specifying the verb, resource, and primary action. It distinguishes itself from siblings (describe_carrier, list_carriers) which are about carrier info, not label creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use optional parameters like 'carrier' (overrides request value) and 'config' (for ad-hoc testing without env vars). It does not explicitly state when not to use the tool or compare to alternatives, but the sibling tools are clearly different in purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_carrierA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses what is returned (required config keys, supported services, label formats) and implies read-only behavior. Does not mention side effects, auth needs, or errors, but is sufficient for a describe tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Purpose and usage are front-loaded. Highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so description need not detail return format. It covers what info is returned and usage hint. Lacks mention of alternative sibling 'list_carriers', but overall complete for a simple describe tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for parameter, but description explains that if 'carrier' is given, carrier-specific details are returned; otherwise, canonical request. Adds meaning beyond bare schema, though could clarify null behavior more explicitly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it describes the canonical shipment request and carrier-specific details if carrier is given. Verb 'describe' and resource 'shipment request and carrier config' are specific. Distinguishes from sibling 'create_label' by advising to call before it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this before `create_label` to learn exactly what to pass,' providing clear when to use. Does not mention when not to use or alternatives, but context with 'list_carriers' suggests this is for detailed info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.