Server Quality Checklist

Tool Scores

• create_partner_referralD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but fails completely. 'Create' implies a write/mutation operation, but there's no information about permissions required, side effects, error conditions, rate limits, or what happens after creation. This is inadequate for a tool with complex nested parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While technically concise with just three words, this is under-specification rather than effective brevity. The description fails to provide necessary information about a tool with complex parameters and no annotations. Every word should earn its place, but here the words don't provide meaningful guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 nested parameters, no annotations, no output schema), the description is completely inadequate. It doesn't explain what the tool does, when to use it, what parameters mean, or what to expect as output. This leaves the agent unable to properly select or invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 3 complex nested parameters, the description provides zero information about what individual_owners, business_entity, or email represent. The schema shows these are required objects with specific structures, but the description doesn't even hint at their purpose or relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a partner referral' is a tautology that restates the tool name without adding meaningful specificity. It doesn't explain what a 'partner referral' is, what resource it creates, or how it differs from sibling tools like create_invoice or create_product. The purpose remains vague despite the clear verb 'Create'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides absolutely no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, context, or comparisons to sibling tools like create_referenced_payout or create_web_profile. This leaves the agent with no information about appropriate use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_referenced_payoutD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but fails completely. It doesn't indicate whether this is a read-only or destructive operation, what permissions might be required, whether it has side effects, rate limits, or what happens upon execution. The single word 'create' suggests a write operation but provides no behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While technically concise with only three words, this represents under-specification rather than effective brevity. The description fails to provide necessary information that would help an agent understand and use the tool correctly. Every sentence should earn its place, but here the single 'sentence' doesn't earn its place by providing meaningful guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the input schema (with nested objects for payout amounts), absence of annotations, lack of output schema, and presence of similar sibling tools, the description is completely inadequate. It provides no context about what the tool does, when to use it, what parameters mean, or what behavior to expect - leaving the agent with insufficient information to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, meaning none of the parameters are documented in the schema itself. The description provides absolutely no information about the 'referenced_payouts' parameter or its complex nested structure. The agent would have no semantic understanding of what data to provide or what 'reference_id', 'reference_type', 'payout_amount', or 'payout_destination' mean.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a referenced payout' is a tautology that essentially restates the tool name without adding meaningful context. It doesn't explain what a 'referenced payout' is, how it differs from the sibling 'create_payout' tool, or what specific action the tool performs beyond the generic 'create' verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides zero guidance about when to use this tool versus alternatives. With sibling tools like 'create_payout' and 'create_payment' available, there's no indication of what distinguishes this tool's purpose or appropriate use cases. No prerequisites, constraints, or comparative context is mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_paymentD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers nothing beyond the basic action. It doesn't indicate whether this is a live transaction, a draft, a test payment, what permissions are required, whether it's reversible, what happens on failure, or any rate limits. For a financial transaction tool with significant implications, this complete lack of behavioral context is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is maximally concise with just two words. While this represents severe under-specification rather than ideal conciseness, from a pure structural perspective, there's no wasted language or unnecessary elaboration. Every word (both of them) directly relates to the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the input schema (3 required parameters with nested objects for payment processing), the absence of annotations, and no output schema, the description is completely inadequate. A payment creation tool requires significant contextual information about authentication requirements, transaction behavior, error handling, and parameter usage that is entirely missing here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning none of the 3 parameters (intent, payer, transactions) or their complex nested structures are documented in the schema. The description adds zero information about what these parameters mean, their expected formats, or how they should be used. This leaves the agent with no semantic understanding of the required inputs for a financial transaction.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a payment' is a tautology that merely restates the tool name without adding any meaningful context. It doesn't specify what type of payment, through which system, or what resources are involved. While it includes a verb ('Create') and resource ('payment'), it lacks the specificity needed to distinguish this from sibling tools like create_invoice or create_order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides absolutely no guidance about when to use this tool versus alternatives. With multiple sibling tools for creating financial transactions (create_invoice, create_order, create_payment_token, create_payout), the agent receives no indication of what distinguishes this payment creation tool from those other options or when it would be the appropriate choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_payment_tokenD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action 'create' without any details about permissions required, whether this is a sensitive operation (handling payment data), rate limits, error conditions, or what happens after creation (e.g., token storage, expiration). This is inadequate for a tool that likely involves financial transactions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, making it front-loaded and free of unnecessary elaboration. However, this conciseness comes at the cost of being under-specified, but as per the scoring rules, it earns full points for brevity and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (payment-related tool with nested parameters), lack of annotations, 0% schema coverage, and no output schema, the description is severely incomplete. It fails to explain what a payment token is, how to use it, what the inputs mean, or what to expect as output, making it inadequate for safe and effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning parameters are undocumented in the schema. The description adds no information about the parameters (customer and payment_source), their purposes, or how they should be structured (e.g., customer ID format, card details). For a tool with 2 required parameters and nested objects, this leaves critical semantics unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a payment token' is essentially a tautology that restates the tool name without elaboration. It specifies the verb 'create' and resource 'payment token', but provides no additional context about what a payment token is, how it differs from other payment-related tools (like create_payment), or what system it operates within. This leaves the purpose vague compared to sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives like create_payment or create_invoice. It does not mention prerequisites, context, or exclusions, leaving the agent with no information to make an informed choice among the payment-related sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_disputeC
  2.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('Get details') without revealing any behavioral traits such as required permissions, rate limits, error handling, or response format. For a read operation with zero annotation coverage, this leaves critical gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, straightforward sentence with no wasted words, making it appropriately concise. However, it is front-loaded only in the sense that it states the core action immediately, but the brevity comes at the cost of completeness, as it omits necessary details for effective tool use.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 1 parameter), lack of annotations, no output schema, and low schema description coverage, the description is incomplete. It fails to address key contextual elements such as what details are returned, error conditions, or dependencies, leaving the agent with insufficient information for reliable operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, meaning the schema provides no semantic context for 'dispute_id'. The description does not compensate by explaining what a dispute_id is, its format, or where to obtain it. This lack of parameter semantics hinders the agent's ability to invoke the tool correctly with appropriate inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get details of a dispute' is a tautology that essentially restates the tool name 'get_dispute' without adding meaningful specificity. It uses a generic verb 'Get' and resource 'dispute' but fails to distinguish what type of details are retrieved or how this differs from potential sibling operations. While it minimally indicates a read operation, it lacks the specificity needed for clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., needing a valid dispute_id), exclusions, or contextual cues for selection among siblings like 'create_payment' or 'list_products'. Without such information, agents must infer usage solely from the name and schema, which is inadequate for informed tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_invoiceC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Create a new invoice' implies a write/mutation operation but provides no information about permissions required, whether the invoice becomes immediately active, what happens on failure, rate limits, or what the response contains. For a creation tool with 5 required parameters and no annotations, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is maximally concise at just three words. There's no wasted language or unnecessary elaboration. While this conciseness comes at the cost of completeness, the description itself is perfectly structured and front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with 5 required parameters (including a complex 'items' array with nested objects), 0% schema description coverage, no annotations, and no output schema, the description is woefully incomplete. It doesn't explain what constitutes a valid invoice, what happens after creation, error conditions, or provide any parameter guidance - leaving the agent to guess about critical aspects of the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning none of the 5 parameters have descriptions in the schema. The tool description provides no information about what parameters are needed or their meanings. The agent must infer everything from parameter names alone, which is inadequate for parameters like 'items' with complex nested structure or 'currency_code' which likely has format requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a new invoice' clearly states the action (create) and resource (invoice), which is better than a tautology. However, it doesn't differentiate this from sibling tools like 'create_order' or 'create_product' - it's generic rather than specific about what makes invoice creation unique compared to other creation operations in this system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With multiple 'create_' sibling tools available (create_order, create_product, create_payment, etc.), there's no indication of when invoice creation is appropriate versus other creation operations, nor any mention of prerequisites or constraints for using this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_web_profileC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Create' which implies a write/mutation operation, but doesn't mention permissions needed, whether it's idempotent, rate limits, or what happens on success/failure. This leaves significant gaps for an agent to understand how to use it safely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just four words, with no wasted language. It's front-loaded with the core action and resource. While it may be too brief for completeness, it earns full marks for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters with nested objects, 0% schema coverage, no output schema, no annotations), the description is insufficient. It doesn't explain what the tool returns, how parameters interact, or provide any context about the web experience profile domain. The agent would struggle to use this effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning none of the 4 parameters have descriptions in the schema. The tool description adds no information about what parameters are needed or what they mean (name, presentation, input_fields, flow_config). For a tool with complex nested objects and no schema documentation, this is inadequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a web experience profile' clearly states the action (create) and resource (web experience profile), which is better than a tautology. However, it's somewhat vague about what a 'web experience profile' entails and doesn't differentiate from sibling tools like create_invoice or create_product, which also create resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are multiple sibling tools for creating different resources (e.g., create_invoice, create_order), but no indication of when a web experience profile is appropriate or what context it applies to.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_userinfoC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implies a read operation ('Get') but doesn't disclose behavioral traits like authentication needs (beyond the token parameter), rate limits, error conditions, or what happens if the token is invalid. The description is minimal and lacks essential operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action ('Get user info') and adds necessary context ('from identity token'). Every part earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple read operation) but lack of annotations and output schema, the description is incomplete. It doesn't explain what 'user info' returns, error handling, or token validation. For a tool with no structured support, more context is needed to be fully helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'identity token' which loosely relates to the 'access_token' parameter, but doesn't explain the parameter's semantics (e.g., format, source, or how it's used). No additional meaning is provided beyond the schema's basic type information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose ('Get user info') and the source ('from identity token'), which is clear but vague. It doesn't specify what 'user info' includes or how it differs from sibling tools, which are all unrelated to user info (e.g., create_invoice, get_dispute). The verb 'Get' is generic but appropriate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid token), exclusions, or context for usage. With sibling tools focused on payments and disputes, this tool stands alone, but no explicit usage context is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_productsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'List all products' implies a read-only operation, but it doesn't specify whether this requires authentication, how data is returned (e.g., format, pagination details beyond schema), or any rate limits. This is a significant gap for a tool with parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, front-loaded and zero waste. It efficiently communicates the core action without unnecessary elaboration, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters with 0% schema coverage, no annotations, and no output schema, the description is incomplete. It doesn't explain the return values, pagination behavior, or any constraints, leaving critical gaps for an AI agent to understand how to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning parameters 'page_size' and 'page' are undocumented in the schema. The description adds no information about these parameters—it doesn't mention pagination, default values, or how they affect the listing. This fails to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all products' clearly states the verb ('List') and resource ('products'), providing a specific purpose. However, it doesn't differentiate from potential sibling tools like 'create_product' beyond the obvious action difference, lacking explicit scope or filtering distinctions that might exist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context for listing products (e.g., after creation or for selection), or comparisons to other tools like 'create_product', leaving usage entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_orderC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this requires specific permissions, has side effects like charging money, involves rate limits, or what the response entails, which is inadequate for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, making it highly concise and front-loaded. Every word contributes directly to stating the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, 0% schema coverage, and no output schema, the description is incomplete. It lacks details on behavior, parameters, and expected outcomes, failing to provide sufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but adds no parameter information. It doesn't explain 'intent' (CAPTURE vs AUTHORIZE) or 'purchase_units', leaving semantics unclear beyond the schema's structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('new order in PayPal'), making the purpose evident. It doesn't explicitly differentiate from siblings like 'create_invoice' or 'create_payment', but the specificity to 'order' provides some distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'create_invoice' or 'create_payment'. The description lacks context about prerequisites, such as authentication or account setup, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_payoutC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'Create a batch payout' implies a write/mutation operation but doesn't mention critical aspects like required permissions, whether this initiates actual financial transactions, error handling, or what happens on success/failure. For a financial tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three words, with zero wasted language. It's front-loaded with the core action and resource, making it immediately scannable and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a financial transaction tool with 2 complex nested parameters, 0% schema coverage, no annotations, and no output schema, the description is severely inadequate. It doesn't explain what a 'batch payout' entails, what the parameters represent, what happens when invoked, or what to expect in return.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning none of the parameters are documented in the schema. The description doesn't mention any parameters at all, failing to compensate for the complete lack of schema documentation. The agent must infer parameter meanings from property names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a batch payout' clearly states the verb ('Create') and resource ('batch payout'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'create_referenced_payout' or 'create_payment', which could cause confusion about when to use this specific tool versus alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_referenced_payout' or 'create_payment'. There's no mention of prerequisites, appropriate contexts, or exclusions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_productC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states 'Create' which implies a write/mutation operation, but doesn't address permissions needed, whether the operation is idempotent, rate limits, or what happens on success/failure. This is inadequate for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized and front-loaded with the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 6 parameters (4 required), 0% schema description coverage, no annotations, and no output schema, the description is severely incomplete. It doesn't explain what the tool returns, error conditions, or provide any context beyond the basic action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning none of the 6 parameters have descriptions in the schema. The tool description provides no information about any parameters, their meanings, or usage examples. It doesn't compensate for the complete lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create a new product') and the target system ('in PayPal'), which provides a specific verb+resource combination. However, it doesn't differentiate this tool from sibling tools like 'create_invoice' or 'create_order' that also create PayPal resources, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_products' or other creation tools. There's no mention of prerequisites, context, or exclusions, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.