Server Quality Checklist

Tool Scores

• create_checkout_payloadA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. It discloses the tool generates form data and hash, and returns specific outputs. However, it does not clarify if this is a pure computation or involves external calls, nor does it specify required authorizations or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, well-structured sentence that front-loads the verb 'Generates' and lists outputs. No redundant information, but could be more concise by splitting into two sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description outlines return types (action URL, form fields, HTML snippet), which is helpful. It implies usage (POST the form data) but does not address prerequisites like merchant credentials. Overall adequate for a moderately complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (88%), so the description adds little beyond schema. It mentions MD5 hash but doesn't tie it to a specific parameter. Baseline 3 is appropriate as the schema already documents parameters well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool generates form data for posting a checkout request, including MD5 hash, and specifies outputs (action URL, form fields, HTML snippet). This distinguishes it from siblings like generate_signature or issue_refund.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No mention of prerequisites (e.g., requiring API credentials) or exclusions. The description only states what it does, not when it's appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_paymentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It specifies the types of payment attempts returned (success, refunded, chargedback), which adds transparency. However, it does not discuss permissions, rate limits, or whether the data is mutable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no superfluous information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one parameter, no output schema, no annotations), the description covers the essential return value and API limitation. It is adequately complete for a straightforward retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% as there is only one parameter. The description repeats the schema's description ('order_id used when initiating the payment'), adding minimal additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns all payment attempts (success, refunded, chargedback) for a given order_id. This is distinct from sibling tools like issue_refund or create_checkout_payload.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states that PayHere does not support listing by date range or status—only by order_id. This provides clear guidance on when to use the tool and what not to expect, though it does not mention alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• issue_refundA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover all behavior. It states returns refund status, but does not detail potential errors, side effects, or idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Minimally complete: explains return value but lacks context on error handling, prerequisites, or performance implications. No output schema to rely on.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by clarifying the conditional use of 'amount' (omit for full, set for partial refund), beyond schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Refunds' and the resource 'payment by payment_id', distinguishing it from sibling tools like create_checkout_payload or get_payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to omit or set 'amount' for full vs partial refund, but lacks explicit instructions on when not to use or alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_signatureA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It reveals that the tool uses a merchant secret from the server environment and never requires it in arguments. It describes conditional behavior for notify mode with verification. However, it does not discuss side effects or performance, but as a pure computation, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loads the core purpose, and contains no filler. Every sentence provides essential information about modes, verification, and security. It is highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers modes and verification well, it does not specify the return type or structure. For a tool with no output schema, describing whether the output is a hash string, a boolean, or an object would be helpful for an AI agent to interpret results correctly. This omission makes it slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 89%, so the schema already documents most parameters. The description adds value by explaining conditional requirements (e.g., amount and currency for checkout, payhereAmount/Currency/statusCode for notify) and the role of expectedMd5Sig. This goes beyond the schema's individual descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool computes a PayHere MD5 hash for checkout submission or notify URL validation, using specific verbs and resources. It distinguishes between two modes and a verification feature, making its purpose unambiguous and distinct from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each mode (checkout vs notify) and how to verify incoming signatures by passing expectedMd5Sig. It also warns against including the secret in arguments. However, it does not explicitly contrast with sibling tools, though the different operations imply appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_credentialsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that the tool checks env vars and OAuth token fetch, indicating a read-only, non-destructive behavior. Does not detail error conditions or output format, but is sufficient for a health check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first defines functionality, second provides usage guidance. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a zero-parameter health check tool. It explains purpose and usage. Could mention expected response format, but overall adequate given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so no additional semantic information is needed. Baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a health check for credentials, specifying the verb 'confirm' and the resource 'env vars' and 'OAuth token'. It distinguishes from sibling tools like create_checkout_payload or issue_refund.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states it is useful as a first call when setting up an integration, providing clear usage context. Does not mention when not to use, but sibling tool names imply different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.