Server Quality Checklist

Tool Scores

• create_mobile_account_paymentA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds 'Money-moving and non-idempotent' and specifies return fields (pp_ResponseCode/pp_ResponseMessage). The annotations already declare non-idempotent and non-read-only, so this adds useful context beyond the annotations, without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loading the core purpose. Every sentence adds value, though it could briefly mention the output structure for completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters and no output schema, the description lacks detail on the full return structure and prerequisites. It does not explain potential errors or safety considerations for a money-moving operation, making it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are described in the schema (100% coverage), so the description adds little beyond clarifying that amount is in PKR and that MWALLET uses CNIC digits. This is adequate but not transformative.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the specific resource 'CNIC-enabled JazzCash Mobile Account payment', including the MWALLET detail and last 6 CNIC digits. This distinguishes it from sibling tools like create_payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives like create_payment or refund_payment. It mentions 'Money-moving and non-idempotent' but does not clarify context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_voucherA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds 'money-moving and non-idempotent,' confirming annotations. It also reveals the voucher is redeemed offline (no redeem API) and that the response includes a voucher number, providing behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences: first defines purpose and currency, second explains workflow and idempotency. No fluff, front-loaded with key info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a payment tool with 6 params (3 required) and no output schema, the description gives essential output hint (voucher number) and offline redemption flow. Lacks error handling or permission details, but adequate given context signals.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions. Description only adds 'amount is in PKR rupees' which is redundant with schema's 'Voucher amount in PKR rupees.' No new semantic value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool creates a JazzCash OTC voucher payment, with specific verb and resource. It distinguishes from sibling tools like create_payment or create_mobile_account_payment by specifying 'Over-The-Counter (OTC) voucher'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for one-time offline voucher payments and notes non-idempotency, but does not explicitly compare to siblings or state when not to use this tool. Lacks explicit when-to-use vs alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refund_paymentA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true (irreversible). The description reinforces this and adds context about automatic paisa conversion and transaction type limitations, which is helpful beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the core action and key constraints. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only two parameters and annotations cover safety, the description is fairly complete. It adds applicability notes but lacks any indication of return output, which is acceptable as there is no output schema defined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description mentions 'amount is in PKR rupees' which is already in the schema, adding no new semantic value. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool refunds a JazzCash transaction and specifies the currency (PKR rupees). The action and resource are unambiguous, but it doesn't explicitly distinguish from sibling tools like create_payment, which is acceptable given the naming.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises that refunds are irreversible and provides a caveat about applicability to card transactions only, urging verification for wallet/OTC. This gives reasonable guidance on when to use and what to check, though it does not specify alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_payment_statusA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Consistent with annotations (readOnlyHint=true), adds detail about returned fields (pp_ResponseCode/pp_ResponseMessage) and idempotent nature. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundant words, front-loaded with action and context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Single-parameter tool with annotations; description covers purpose, read-only nature, and return content adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes txn_ref with 100% coverage; description adds no new parameter info beyond confirming reference usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb ('look up'), resource ('JazzCash transaction status'), and method ('by its reference'). Distinguishes from sibling tools like create_payment and refund_payment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for status inquiry vs. creation/refund in siblings, but lacks explicit when-not or alternatives beyond stating 'Read-only'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_paymentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors beyond annotations: amount conversion to paisa, auto-generated txn_ref, return fields (pp_ResponseCode/pp_ResponseMessage), and non-idempotency warning. No contradiction with annotations (readOnlyHint=false, idempotentHint=false).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: first states purpose, second clarifies key parameter behavior, third warns about idempotency. No wasted words, front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all essential aspects: purpose, parameter conversion, idempotency, response format, and non-idempotency warning. For a 5-parameter tool with no output schema, this description provides comprehensive context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds value by explaining amount unit (PKR to paisa), txn_ref auto-generation, and response details. However, some info repeats schema descriptions (e.g., 'converted to paisa automatically').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool creates a JazzCash Mobile Wallet payment. Specifies verb 'Create' and resource 'JazzCash Mobile Wallet (MWALLET) payment'. Distinguishes from siblings like 'check_payment_status' and 'refund_payment' by focusing on MWALLET creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit warning: 'Money-moving and non-idempotent — do not retry blindly.' Also notes txn_ref auto-generation, guiding use when idempotency is needed. Lacks explicit mention of when to use versus siblings like 'create_mobile_account_payment'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.