Server Quality Checklist

Tool Scores

• chainpay_list_ordersA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions pagination and status filter but fails to disclose key behaviors such as default sort order, result set scope (user-specific or global), rate limits, or permission requirements. This is minimal and leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 9 words, perfectly front-loaded and efficient. Every word earns its place, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (list with pagination and filter), the description is adequate for basic use. However, it lacks details on default values, output format, error handling, and sorting order. For a complete understanding, the agent may need to infer or experiment.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all three parameters. The description adds no extra meaning beyond the schema; it simply reiterates 'pagination and optional status filter'. Baseline of 3 is appropriate for high schema coverage without additional insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists payment orders with pagination and optional status filter. It uses a specific verb and resource, distinguishing it from sibling tools like chainpay_create_order (create) and chainpay_get_order (single order).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing orders but does not explicitly state when to use this tool versus alternatives (e.g., chainpay_get_order for a single order). No exclusions or when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chainpay_get_orderA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It indicates a read operation without destructive effects, but does not disclose behaviors like error handling, authentication, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, front-loaded with verb and resource. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks output schema, description does not mention what 'details and current status' include (e.g., status, amount, timestamps). Minimal but adequate for a simple read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with clear parameter description. The tool description adds no additional meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'get' and resource 'specific payment order', distinguishing it from sibling tools like list_orders (multiple orders) and create_order (creation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. Purpose is clear enough for simple cases, but lacks differentiation advice vs similar tools like chainpay_list_orders.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chainpay_get_balanceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses the returned balance components, which is useful, but does not mention whether the operation is read-only, any authorization requirements, or rate limits. The basic behavior is described but with gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately states the tool's purpose and key details, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description adequately lists the three balance components returned. However, it could mention the currency or format to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so baseline is 4. The description adds no extra meaning beyond the schema, which is acceptable for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Get' explicitly naming the resource 'merchant account balance' and lists the specific sub-balances (total, frozen, available), clearly distinguishing this from sibling tools which deal with orders, withdrawals, and chains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While the context makes it fairly obvious, the description does not include any explicit usage directions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chainpay_create_withdrawalA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that admin approval may be required, which is a key behavioral trait beyond a simple 'create' operation. However, it does not specify return format or error behavior, but for a submission tool this is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short sentences, with the main purpose front-loaded in the first sentence. It is efficient, contains no unnecessary text, and each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description provides the essential purpose and conditions. However, it lacks information about the response (e.g., whether a withdrawal ID is returned) and error scenarios. It is adequate but not fully comprehensive for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all three parameters with complete descriptions (100% coverage). The description adds little beyond the schema, only implying the amount and token symbol form. Since the schema already documents the parameters well, the description's marginal value is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Submit a withdrawal request'), the resource ('funds to the merchant's bound withdrawal address'), and distinguishes from sibling tools like chainpay_create_order (order creation) and chainpay_get_balance (balance query). It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes prerequisites ('Requires a withdrawal address to be configured') and a condition ('May require admin approval'), guiding the agent on when the tool is applicable. It does not explicitly mention alternatives or when not to use, but the context and sibling names make the use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chainpay_create_orderA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the creation action, return values, and supported chains, but does not disclose rate limits, authentication needs, or potential side effects like fund holds.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose, return info, supported chains, and a prerequisite. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description mentions return values (payment URL and wallet address) and a prerequisite. For a creation tool with 6 parameters, it covers essential aspects, though error handling or idempotency details are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter described. The description adds value by noting that the chain parameter corresponds to TRON/BSC/POLYGON and that minimum amounts should be checked, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create a new cryptocurrency payment order' and specifies it returns a payment URL and wallet address. It distinguishes from siblings like chainpay_create_withdrawal by describing a different action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance to 'Call chainpay_get_supported_chains first to check minimum amounts' provides a clear prerequisite. While it does not explicitly state when not to use, the sibling names and context imply distinct use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chainpay_get_supported_chainsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It explicitly states this is a read operation ('Get all ...') and discloses the output includes fee rates and minimum amounts. It doesn't mention potential rate limits or pagination, but for a simple list tool with no parameters, this is adequate. A higher score would require more detail on edge cases or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first defines purpose and output, the second gives usage guidance. No unnecessary words. Front-loaded with the main action. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description provides essential information: what data is returned (chains, tokens, fees, minimum amounts) and when to use it (before creating orders). This is sufficient for an agent to understand and correctly invoke the tool. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty. Schema coverage is 100% by definition. Description cannot add parameter semantics because there are none. Baseline for 0 params is 4, and the description does not detract from this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets all supported blockchain networks and tokens, including fee rates and minimum amounts. It uses a specific verb (Get) and identifies the resource (supported chains/tokens). This distinguishes it from sibling tools like chainpay_create_order which perform actions on the data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Call this before creating orders to get valid chain/token values.' This provides clear guidance on when to use the tool and its role as a prerequisite for order creation. While it doesn't mention withdrawals, it covers the primary use case effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.