Server Quality Checklist

Tool Scores

• historyA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, the description discloses that it spends no money, that the data source differs by mode, and that results are newest-first. This adds meaningful behavioral context that the annotations alone do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core behavior and is information-dense without wasted words. Each clause contributes: mode behavior, ordering, safety, and use cases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only two optional parameters fully covered by the schema and annotations covering safety properties, the description supplies the missing decision-relevant context: what data is shown, where it comes from, ordering, and when to use the tool. Nothing critical is missing for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so both the agent and limit parameters are already well documented. The description does not add parameter-specific detail, but it does provide surrounding context about delegated vs. self-custody modes, which weakly relates to the agent parameter. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it lists recent payments. It further clarifies the two modes (server-side feed vs. locally recorded receipts) and provides concrete use cases, making its purpose unambiguous and distinct from sibling tools like balance or pay.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use the tool: to reconcile payments, retrieve a past receipt, or audit spending. It does not explicitly name alternatives or state when not to use it, but the context is clear enough for an agent to select it correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspectA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, the description adds meaningful operational detail: it spends no money, sends no payment header, needs no API key or account, and is always safe. This goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. The core behavior and cost-relevant outputs are front-loaded, followed by safety guarantees and the explicit routing to 'pay'. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, read-only inspection tool with rich annotations, the description covers what the tool does, what it returns conceptually, and when to invoke it. No important behavioral or usage information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the single 'url' parameter with a clear description, so the baseline is 3. The tool description reinforces that the URL should be x402-protected but adds little parameter-specific meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Fetches'), a specific resource ('x402-protected URL'), and the key scope ('WITHOUT paying'). It clearly differentiates from the sibling 'pay' by emphasizing that this tool reports price and requirements without spending money.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells the agent to use this before 'pay' to know costs, which is clear usage context. It does not mention other sibling tools like 'check_payment_signal' or 'discover' as alternatives, so it lacks full when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• balanceA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive, and the description reinforces this with 'never pays.' Beyond annotations, it adds rich context about the two identity modes (delegated ArisPay agent vs self-custody key-derived address), what data is returned, and the concept of spend-mandate limits, so an agent knows what behavior to expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler: the first establishes safety and output scope, the second gives concrete use cases. Every clause contributes information and the critical 'Read-only / never pays' signal is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter, rich annotations, and no output schema, the description covers safety, returned content, and when to use it. The only notable gap is the exact default behavior when the agent parameter is omitted, which would make the identity-selection semantics fully unambiguous.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the optional agent parameter with 100% coverage. The description adds meaning by relating a stored agent to delegated custody and the PAYAGENT_PRIVATE_KEY to self-custody, which helps clarify the parameter's role. However, it does not explicitly state what happens when the agent parameter is omitted versus provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Read-only: shows the active payment identity' and enumerates exactly what is returned: USDC deposit address, on-chain balance, and spend-mandate limits. It also links the tool to concrete tasks like finding where to send USDC and confirming a deposit, making it easy to distinguish from siblings such as pay, history, or check_payment_signal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly gives three trigger conditions: find where to send USDC, confirm a deposit landed, and diagnose a failed payment. It does not name alternative sibling tools or state when not to use it, so it falls short of a 5, but the usage context is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_payment_signalA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description thoroughly discloses behavior beyond the annotations: real-money spend per uncached call, free cached/unknown responses, network and payment side effects, verification against published JWKS, and the caveat that the result is observational evidence, not a guarantee. This meaningfully enriches the annotation hints and contradicts nothing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but almost every sentence earns its place given the unusual cost model and decision heuristic. It is front-loaded with the most critical information (SPENDS REAL MONEY) and organized from cost to functionality to usage to side effects. Slight over-density keeps it from a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema or nested object schema, the description explains what the tool returns (signed arispay.signal.v1 artifact with explicit freshness), how it is verified, the exact side effects, the caching behavior, and the usage threshold. This is complete enough for an agent to call it correctly and interpret its result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description reinforces that the resource must be the exact absolute HTTPS URL because evidence is keyed by that URL, but it does not add meaningfully new parameter-level semantics beyond what the schema states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: check_payment_signal buys and returns ArisPay Signal evidence about an x402 resource before payment. It clearly distinguishes this from siblings by framing it as a pre-payment verification step ('Use it BEFORE a material first payment') rather than the payment itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use and when-not-to-use guidance: use before material first payments or after cached evidence expires, and skip when cumulative exposure is at or below the 1¢ check price. It also explains the decision denominator (cumulative exposure vs. cost of one failure) with a concrete example.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discoverA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool readOnly, openWorld, idempotent, and non-destructive. The description adds meaningful context beyond annotations: it states 'Read-only — spends no money, needs no API key' and discloses the return shape ('ranked candidates with endpoint URL, price in cents, health and verification flags'). No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states the core purpose, and the second gives the operational safety and output summary. The final sentence provides clear workflow guidance. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only discovery tool, the description covers what is searched, optional budget constraints, safety, output contents, and the correct next tool to use. The input schema fully documents parameters, and annotations cover safety, so no critical context is missing even though there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so query, limit, category, and budgetCentsMax are already fully documented by the input schema. The description adds only a plain-language framing and the budget bound, which does not significantly exceed the schema's own descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Searches') and resource ('ArisPay marketplace'), then defines the search style as 'capability or plain-language intent.' It also distinguishes discover from its siblings by naming the find → inspect → pay workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to use the tool: 'Select this to FIND an endpoint; then inspect to price a specific URL, then pay to call it.' This positions discover as the exploration step and clearly separates it from subsequent sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• payA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description discloses irreversibility, the 402-triggered payment flow, idempotency-key caching behavior, local vs. delegated signing, and balance/mandate constraints. These details are consistent with destructiveHint=true and idempotentHint=true; no contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense yet front-loaded with the most important warning ('SPENDS REAL MONEY'), then the workflow, idempotency behavior, mode selection, and usage guidance. Every sentence carries necessary information without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description clearly explains the complete payment cycle, idempotency guarantees, both execution modes, and the shape of the result (response plus machine-readable receipt). It is thorough enough for an agent to invoke it correctly and safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers only 50% of parameters with descriptions, and the description strongly compensates by thoroughly explaining idempotencyKey, agent mode, and the payment path. Method, headers, and body are not detailed in prose, but they are standard HTTP concepts and largely self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action with a critical warning: it spends real money, handles the x402 payment flow, retries the request, and returns response plus receipt. It clearly distinguishes itself from siblings by mentioning 'Use inspect first' and 'Do not use for ordinary unpaid HTTP requests.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells the agent to call inspect first for pricing and warns not to use this for ordinary unpaid HTTP requests. It also explains the two modes of operation and the server-side mandate restrictions, giving clear when-to-use versus when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• setupA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already carry idempotentHint=true, readOnlyHint=false, destructiveHint=false, and openWorldHint=true. The description goes further by itemizing side effects (registers email, creates Coinbase CDP-managed wallet, writes credentials to ~/.payagent/config.json), stating that no money moves, and explaining the idempotent recovery behavior. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences: purpose and side effects, money-safety, idempotency, and the self-custody exclusion. Every sentence earns its place, and the main verb and resource are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and only two simple parameters, the description covers what happens, side effects, safety, idempotency, and when to skip it. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents name and email. The description adds that email is also the recovery key and advises 'use a real address,' which is critical context for correct parameter choice. It does not add much about the optional name, but the schema already covers its fallback behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Creates or recovers an ArisPay account for delegated custody and provisions a USDC payer wallet in one call.' It enumerates concrete side effects and clearly distinguishes setup from siblings like pay, history, and balance by framing it as the account/wallet provisioning step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when not to use: 'if PAYAGENT_PRIVATE_KEY is set, skip setup entirely — pay works with the local key alone.' This routes the agent to the pay sibling and self-custody mode. It also implies the right context for setup (delegated custody).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.