Server Quality Checklist

Tool Scores

• search_productsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not disclose whether this is a read-only operation, how results are ordered, pagination behavior, or any rate limits. The description only states what the tool does, not its behavioral nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the purpose and lists filters without any wasted words. It is concise and structured for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description fails to explain the return format (e.g., a list of product objects), pagination, or the semantics of the limit parameter. An agent calling this tool would not know what to expect in response or how to use limit effectively, making it incomplete for a 5-parameter search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 40% (q and brand have descriptions; limit, country, and category do not). The description lists the filter fields (q, brand, country, category) but omits limit entirely. It adds some context by clarifying these are filters, but does not explain formats like country code length or what category expects, leaving gaps for undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Search/list') and resource ('purchasable products'), and lists the available filter dimensions. It doesn't explicitly differentiate from siblings like get_product or list_brands, but the purpose is unambiguous enough for an agent to recognize it as a search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is for searching products, but provides no explicit guidance on when to use it versus alternatives (e.g., get_product for a single item, list_brands for brands). No when-not-to-use or alternative routing is given, leaving the decision to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_orderA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description must disclose behavior. It does reveal that it creates an order, returns an invoice, and that payment is required for the reveal link. However, it doesn't mention idempotency behavior, failure modes, authentication requirements, or the full return format. For a mutation tool with no annotations, more disclosure is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each carrying information: the action, the follow-up step, and the dependency. No filler, front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that creates an order and returns an invoice, the description covers the flow: create, pay, poll. It mentions the dependency and the next step. With no output schema, it indicates the primary return (invoice). It lacks details on error handling or idempotency, but the essential workflow is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 67% of parameters with descriptions. The description adds meaning for variantId by pointing to get_product as its source, which is helpful. It doesn't explain productId or the relationship between amount and currency beyond schema. Overall, it adds some value but doesn't fully compensate for the missing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (create an order), the resource (order), and the outcome (get a Lightning invoice). It distinguishes itself from siblings by naming get_order_status as the next step and get_product as the source of variantId, making its role unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance: use when you want to create an order and get an invoice, and it tells the agent to poll get_order_status after payment. It also specifies a prerequisite (variantId from get_product). It doesn't explicitly state when not to use, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_order_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. The verb 'Check' conveys a read-only operation, and the description specifies the observable outputs. It does not mention error behavior or side effects, but for a simple status query this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the action, with no wasted words. The parenthetical clarifies the reveal link without adding bulk.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema, the description covers the essential return values (payment state, reveal link availability) and implies the input. It lacks explicit parameter guidance and edge-case behavior, but these are minor for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the orderId parameter or its format/meaning beyond the schema's property name. Since the description must compensate for low schema coverage and does not, this is a clear gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Check') and resource ('order'), and clarifies exactly what is returned (payment state, reveal link availability). This clearly distinguishes it from sibling tools like create_order and get_product, which operate on different resources or actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: use this when you need an order's payment status or reveal link availability. It does not explicitly name alternatives or exclusions, but the purpose is unambiguous relative to the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses that the response includes denominations/variants and specifically surfaces the variantId + amount fields relevant to create_order, which is useful beyond the tool name. It doesn't mention errors or auth, but for a simple read-only lookup the key return characteristic is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no redundant words; it leads with the action and resource, then appends the key output detail in a parenthetical. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema and no annotations, the description covers the essential lookup semantics and highlights the output fields that matter for a downstream create_order call. It stops short of full return-format or error details, but the minimal complexity means nothing critical is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents the single id parameter with a description ('giftcardshop product id') and minLength, so schema coverage is 100%. The tool description only repeats 'by id' and adds no new semantic detail about the parameter, matching the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses the specific verb 'Get' with a clear resource ('one product') and lookup key ('by id'), immediately distinguishing it from list_brands and search_products. The added detail about denominations/variants further clarifies the exact resource shape.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this to retrieve a single product by id, and the output supplies variantId and amount needed for create_order. It does not explicitly name sibling tools or state when to prefer search_products, but the direct-lookup framing makes the intended usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_brandsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'list' clearly indicates a read-only operation with no side effects. It does not mention rate limits or data modifications, but none are implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that covers the core functionality and optional parameters without unnecessary fluff. It is well-structured and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool, the description provides sufficient context: it states what is listed, the optional filter, and pagination support. It does not specify the output format (e.g., fields returned), but since no output schema is provided, this is not strictly required for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly mentions country filtering and pagination, giving context for the 'country', 'limit', and 'offset' parameters. However, the schema only provides a description for 'country'; 'limit' and 'offset' lack individual explanations, leaving some ambiguity about their exact semantics (e.g., default values).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: listing gift-card brands from the catalog. It also mentions optional filtering and pagination, distinguishing it from sibling tools that focus on products and orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when brand lists are needed) and explains optional parameters (country filter, pagination). It does not explicitly contrast with sibling tools, but the domain difference is clear from the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.