Server Quality Checklist

Tool Scores

• create_payment_preferenceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates a payment preference and returns a URL, but lacks details on permissions, rate limits, error handling, or whether the operation is idempotent. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and result. It avoids redundancy but could be slightly more detailed given the lack of annotations and schema descriptions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 6 parameters, 0% schema coverage, no annotations, and no output schema, the description is incomplete. It covers the basic purpose and return value but lacks parameter explanations, behavioral details, and usage context, which are critical for effective tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It does not explain any parameters, such as the meaning of 'back_urls' or 'notification_url', leaving all 6 parameters without semantic context beyond the schema's structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Creates a Mercado Pago checkout payment preference') and resource ('payment link'), distinguishing it from siblings like create_refund or get_payment. It also specifies the return value ('init_point URL for redirecting buyers'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like create_refund or search_payments. It mentions the return value but does not specify prerequisites, such as authentication or account setup, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paymentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves payment details, implying a read-only operation, but doesn't address key aspects like authentication requirements, error handling (e.g., for invalid IDs), rate limits, or whether it's idempotent. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that are front-loaded: the first states the core action, and the second specifies the return details. There's no unnecessary information, and it efficiently communicates the essential purpose. However, it could be slightly more structured by explicitly separating usage context from behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple retrieval with one parameter) and lack of annotations or output schema, the description is partially complete. It covers the basic purpose and return scope but misses critical context like error scenarios, authentication, and differentiation from siblings. For a tool with no structured data support, it's adequate but leaves clear gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantic value beyond the input schema. It mentions 'payment_id' as the parameter but doesn't explain its format (e.g., UUID, numeric), source, or constraints. With 0% schema description coverage, the description doesn't compensate for the lack of parameter documentation, though it at least names the parameter. This meets the baseline for minimal viability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve a payment by its ID' specifies the verb (retrieve) and resource (payment), and 'Returns full payment details including status, amount, payer info' indicates the scope of information returned. However, it doesn't explicitly differentiate from sibling tools like 'search_payments' or 'get_merchant_info', which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'get_payment' over 'search_payments' (e.g., for a specific ID vs. filtering), nor does it specify prerequisites or exclusions. This lack of contextual usage information limits its effectiveness for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_paymentsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'recent payments' and 'authenticated merchant', but lacks details on permissions, rate limits, pagination behavior (beyond schema parameters), or what 'recent' means. This is inadequate for a search tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It front-loads the core purpose and includes key constraints, making it appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema description coverage, and no output schema, the description is incomplete. It lacks details on behavioral traits, parameter meanings beyond status, and expected return values, which are critical for a search tool with three parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only mentions 'filtering by status' for one of the three parameters (status, limit, offset), leaving limit and offset undocumented. This adds minimal value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('recent payments for the authenticated merchant'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_payment' or 'create_refund', which might have overlapping domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for searching payments with status filtering, but provides no explicit guidance on when to use this tool versus alternatives like 'get_payment' or 'create_refund'. It mentions the authenticated merchant context, which gives some implied context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_merchant_infoB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this retrieves the 'authenticated merchant's' profile, implying authentication is required, but doesn't specify what happens if authentication fails, rate limits, or response format. For a read operation with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. Every word earns its place: 'Retrieve' (action), 'authenticated merchant's user profile' (target), and 'including ID, nickname, and site' (scope). There's no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain what the return value looks like (structure, data types), error conditions, or authentication requirements. For a tool that presumably returns user profile data, more contextual information would be helpful despite the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the lack of inputs. The description appropriately doesn't add parameter details beyond what's in the schema, maintaining a baseline score of 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve'), target resource ('authenticated merchant's user profile'), and scope of data returned ('including ID, nickname, and site'). It distinguishes itself from sibling tools like get_payment (which retrieves payment data) and search_payments (which searches multiple payments).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like authentication requirements), nor does it differentiate from sibling tools beyond what's implied by the purpose. No explicit when/when-not statements or alternative tool references are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_refundA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the action ('Refund') but does not disclose critical behavioral traits such as whether this requires specific permissions, if refunds are reversible, potential rate limits, or what the response looks like (e.g., success/failure indicators). This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence, followed by a specific usage rule in the second. Every sentence earns its place by providing essential information without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a refund operation (a mutation with no annotations and no output schema), the description is somewhat complete by stating the action and parameter usage. However, it lacks details on behavioral aspects like permissions, reversibility, or response format, which are important for such a tool. It compensates partially with parameter guidance but falls short of full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'payment_id' is required for refunding a payment and that 'amount' is optional (omit for full refund), which clarifies the semantics beyond the basic schema types. However, it does not detail formats (e.g., string format for payment_id) or constraints (e.g., amount must be positive).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Refund a payment') and resource ('payment'), distinguishing it from sibling tools like create_payment_preference, get_merchant_info, get_payment, and search_payments which are about creating preferences or retrieving information rather than processing refunds.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool (for refunding payments) and includes a usage rule (omit amount for full refund), but it does not explicitly state when not to use it or name alternatives among the sibling tools, such as when to use get_payment for checking payment status instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.