Server Quality Checklist

Tool Scores

• query_refund_detailB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavioral traits. It states query intent but lacks disclosure of auth requirements, rate limits, error handling, or implications of invalid IDs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with a clear opening sentence and minimal redundancy, though the Args section duplicates schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given known output schema and simple tool, description covers core purpose and parameter meaning, but omits prerequisites, error states, and usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, description adds meaning by explaining refund_request_id as merchant-assigned unique ID and imposing a 64-character limit, but lacks format or example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'inquire' and resource 'refund status', and clarifies it applies to 'previously submitted refund request', distinguishing it from siblings like create_refund and cancel_payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly indicates use after submitting a refund request, but does not explicitly state when to use this versus siblings like create_refund or query_payment_detail, nor provides exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_payment_sessionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description implies a write operation (creation) but doesn't disclose side effects, idempotency, or required permissions. It adds parameter constraints but lacks deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and then lists parameters clearly. It's concise with no unnecessary words, though it could be slightly more structured (e.g., bullet points for parameters).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a payment tool, the description covers parameter constraints but omits error conditions, prerequisites, or return behavior (though output schema exists). It's adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds constraints (max lengths, currency format, value range) for all 4 parameters, significantly enhancing meaning beyond the schema's title and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a payment session and its benefit of eliminating redirections. It is a specific verb+resource and distinct from sibling tools (cancel, refund, query).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description only states what it does, without explaining when to prefer it over siblings like query_payment_detail or create_refund.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_payment_detailA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states 'inquire' implying a read-only operation, but does not disclose behavioral traits such as what happens if the payment request ID is invalid, authentication requirements, or any side effects. The description is minimal beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences and a structured Args block. It is front-loaded with the purpose and includes parameter documentation without unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to explain return values. However, it lacks information about error handling, prerequisites, or what to do if the payment request ID does not exist. The description is adequate for a simple query tool but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 0%, the description adds meaning to the payment_request_id parameter by stating it is a unique ID assigned by a merchant and specifying a maximum length of 64 characters. This provides context beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is used to inquire about transaction status and other information for a previously submitted payment request, using a specific verb ('inquire') and resource ('transaction status and other information'). It distinguishes from siblings like cancel_payment or create_refund which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for querying previously submitted payment requests but does not explicitly state when to use it versus alternatives (e.g., query_refund_detail) or when not to use it. No exclusions or context for when this tool is appropriate are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_paymentA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist; description adds cancellable period constraint but lacks disclosure of side effects, auth needs, or error scenarios.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences cover purpose, constraint, and param – concise, but the Args block could be integrated for better flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a one-param tool with output schema; missing return value hints or error conditions like duplicate cancellation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description explains payment_request_id's meaning and max length, adding value beyond the schema's title and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool cancels a payment when the result is not returned after a long time, distinguishing it from sibling tools like create_payment_session and create_refund.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies when to use (long timeout) and when not (outside cancellable period), but does not guide against siblings like refund, which could be an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_refundA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses refund limits and window enforcement, but with no annotations, it omits idempotency, side effects, or error handling specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, well-structured, with purpose, conditions, and parameter details in a logical order without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers core behavior and parameter semantics; could mention return value structure or error scenarios, but output schema exists and complexity is low.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Each parameter is described with purpose, format, and constraints (e.g., refund_amount_value as smallest currency unit), adding significant value beyond the schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool initiates a refund against a successful payment, distinguishing it from siblings like cancel_payment or create_payment_session.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides conditions for use (full/partial refund, multiple refunds allowed, refund window) but lacks explicit when-not-to-use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.