Server Quality Checklist

Tool Scores

• approve_pendingB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that approving grants one-time re-run permission and that it cannot change policy. Missing details on deny action behavior, return values, and side effects, but adequate for core function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with main purpose, no redundant information. Efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-param tool with no output schema and no annotations, the description covers purpose, action types, and a key behavioral constraint. Missing details on deny action, return format, and parameter validation, but functional overall.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no parameter descriptions (0% coverage). The description implies action enum values and the role of approvalId for approve/deny, but does not explicitly define each parameter or their requirements. Insufficient compensation for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists or resolves pending payments in queue mode, with specific verb 'list or resolve' and resource 'payments waiting for human approval'. It distinguishes from siblings by focusing on approval workflow, though it doesn't explicitly differentiate from each sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies it only resolves payments already queued for approval and cannot change spending policy, giving clear context. However, it lacks explicit when-not-to-use guidance or alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_receiptsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only indicates a read operation but lacks details on side effects, authorization requirements, rate limits, or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose, and no extraneous information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple query tool, it provides adequate context about filtering. However, it lacks output schema and details about response format or pagination, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaning for 3 of 4 parameters (sinceTs, host, outcome) by linking them to filters. The 'limit' parameter is not described, but its schema includes default and bounds. Some value added, but not fully compensating.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool queries the local payment receipt ledger, a specific verb and resource. It differentiates from siblings by noting spending policy constraints, indicating no sibling tool can widen it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies filters (time, host, outcome) and implies usage for obtaining audit trail information. However, it does not explicitly compare to siblings or state when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• payment_quoteA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. It clearly states the tool does not pay (non-destructive) and that spending policy is operator-owned and not widen-able. Adds crucial behavioral context beyond schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: first states core action with important disclaimer (WITHOUT paying), second enumerates return data, third adds policy context. Front-loaded, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description adequately covers main purpose and return data but lacks details on parameters, error handling, and output format. No output schema provided, so description is helpful but not complete for all 6 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It only implicitly explains 'url' via context; parameters body, chain, method, headers, tokenAddress are not described. Description adds minimal value for parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool checks cost and policy allowance for a paid URL without actually paying. It specifies verb (check), resource (paid URL cost, policy decision), and distinguishes from siblings like paid_fetch (actual fetch) and spend_status (status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage for pre-payment checks but does not explicitly state when to use over siblings or provide exclusions. No mention of alternatives or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paid_fetchA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations present, so description carries full burden. It discloses automatic payment behavior, operator threshold checks, and that free URLs cost nothing. Does not cover failure behavior or rate limits, but adequately covers key behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise, front-loaded sentences with no fluff. Each sentence adds essential information about function, usage guidance, and policy implications.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, yet description does not mention return values. With 9 parameters, the description only covers payment-related semantics. Adequate for core behavior but missing parameter details and output format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 11% (only maxAmountUsd has a description). The description adds little parameter-specific meaning beyond the payment concept. Fails to explain url, method, headers, body, chain, dryRun, responseMode, tokenAddress.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a URL and automatically pays if required, distinguishing it from siblings like payment_quote. It uses specific verbs and resources ('fetch a URL', 'paying if required').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: mentions when to use payment_quote first as an alternative, and explains the spending policy and approval threshold. Lacks explicit exclusions for other sibling tools, but gives solid guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spend_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It clearly states the tool shows information (totals, budgets, holds, payments) and notes that spending policy is operator-owned and cannot be widened by any tool. This adequately informs about its read-only nature and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence efficiently lists what the tool shows, and the second adds a key policy constraint. Information is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although there is no output schema, the description outlines the scope of returned information (totals, budgets, holds, payments) sufficiently for a simple status tool. Given the context of sibling tools and no parameters, the description is complete enough for the agent to understand its use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100% by default. The description does not need to add parameter meaning since none exist; baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it shows today's agent spending, including totals, remaining budgets (overall and per host), active holds, and recent payments. It distinguishes itself from siblings like approve_pending and list_receipts by being a read-only overview tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for viewing spending status but does not explicitly state when to use this tool vs. alternatives or when not to use it. It provides context about the spending policy but lacks explicit guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.