Server Quality Checklist

Tool Scores

• get_transactionB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states the tool 'gets' a transaction, with no disclosure of error states (e.g., what happens if ID not found), rate limits, or idempotency. The behavioral insight is very limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with no redundant or extraneous information. It is appropriately concise for a simple getter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple single-resource fetch with no output schema, the description provides the core purpose but lacks context about error handling, return format, or usage scenarios. It is minimally adequate but not rich.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the two parameters, and the description does not add new meaning beyond the schema's own descriptions. It states the tool is for a Mercury account, which aligns with account_id, but that is already implied by the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a single transaction by ID for a Mercury account, using a specific verb and resource. It effectively distinguishes from siblings like list_transactions (multiple) and get_account (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives such as list_transactions. The description does not specify prerequisites, nor does it mention when not to use it (e.g., for batch retrieval).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_accountA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must carry full burden. Only mentions inclusion of balances, but fails to disclose read-only nature, idempotency, rate limits, or error conditions. Minimal behavioral insight beyond basic output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence conveying purpose and output concisely. No redundant words; every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one required parameter and no output schema, description adequately states purpose and key output (balances). Could be more complete by hinting at other returned fields (e.g., account name, currency), but sufficient for a simple read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with account_id adequately described as 'Mercury account ID (UUID)'. The tool description does not add further meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly specifies the action 'Get', the resource 'single Mercury account', and includes distinct output details 'current and available balances'. Distinguishes from sibling tools like list_accounts (plural) and get_transaction (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when needing a specific account by ID, but provides no guidance on when not to use it (e.g., for listing all accounts) or alternatives. Lacks explicit context of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description includes a 'Read-only' label, which is a behavioral trait. However, with no annotations, it does not disclose authentication needs, response format, or whether the list is paginated. The description is adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and concise with no unnecessary words. It front-loads the key action and information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, the description covers the core functionality but leaves out details like response structure or potential limits. For a simple list tool, this is minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema coverage is 100%. The description adds no parameter information, which is acceptable as there are none. Baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'Mercury bank accounts,' specifying the data returned (balances, account/routing numbers, status). This distinguishes it from sibling tools like 'get_account' (single account) and 'list_transactions' (transactions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'get_account' for a single account. There is no mention of prerequisites, limitations, or context where it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_statementsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the 3-month limit, but lacks details on pagination, error conditions, or authorization needs. The return format is hinted but not fully specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and key constraint. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so description should compensate. It mentions response fields but not pagination or error details. For a 5-parameter tool, it is moderately complete but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add meaning beyond schema; it mentions response fields but not parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'List monthly statements for a Mercury account' with specifics about returned fields (period, ending balance, PDF download URL). It distinguishes from siblings like download_statement_pdf, which downloads instead of lists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage constraint: 'Date range filters are limited to 3 months per request.' While it doesn't explicitly compare to alternatives, the sibling context makes it obvious that list_statements is for listing and download_statement_pdf for downloading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_transactionsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explicitly states 'Read-only', which is good, and notes the default date range. However, it does not disclose potential side effects, error handling, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Clear and front-loaded with the core purpose. Every sentence adds useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic function but lacks details about output format, pagination behavior (though offset parameter exists), and error scenarios. For a list tool without output schema, more context on what is returned would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal value by mentioning filtering by date range, status, and search term, which correspond to parameters start/end, status, and search. No new semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists transactions for a Mercury account, with filtering options like date range, status, and search term. It distinguishes itself from siblings like 'get_transaction' (single) and 'download_statement_pdf' (file download).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions default behavior (last 30 days) and filtering, but does not explicitly state when to use this tool vs alternatives like 'get_transaction' or 'list_accounts'. No exclusion criteria or prerequisites are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_statement_pdfA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions the URL validation but lacks details on file overwriting behavior, error handling (e.g., invalid URL), or authorization requirements. The behavioral disclosure is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. The most critical information (purpose and URL restriction) is front-loaded. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple download tool with two parameters, the description covers the essential: source constraint and destination. It could mention what happens if the output file exists or if the URL fails, but overall it is sufficiently complete given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for both parameters. The description adds extra value by specifying that only mercury.com URLs are accepted, which is not in the schema. This enhances semantic clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (download), the resource (statement PDF), the source (from list_statements downloadUrl), and the destination (local file). It also adds a URL restriction, making the purpose unambiguous and distinguishing it from siblings like list_statements or list_transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear constraint (only mercury.com URLs are accepted) and implies the tool is used after list_statements. However, it does not explicitly state when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.