Server Quality Checklist

Tool Scores

• spending_summaryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It mentions grouping and totals but omits critical behavioral details such as the ability to filter by date range (dateFrom, dateTo) and the default limit and sorting behavior, which are only present in the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, with three clear sentences that front-load the purpose. It avoids unnecessary detail and is easy to parse, though it could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given five parameters and no output schema, the description omits important context such as the meaning of dateFrom/dateTo for filtering and the default limit of 20. It also lacks any hint of the return format beyond 'totals', making it incomplete for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal value beyond the input schema: it reiterates the groupBy options but does not explain the purpose of connectionId, dateFrom, dateTo, or limit beyond what the schema already provides. With 40% schema coverage, the description should compensate more.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool groups expenses by merchant or category with totals, showing where money is spent. It distinguishes from sibling tools like list_transactions and get_balance by focusing on aggregation rather than raw data or balances.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description suggests when to use each groupBy option but does not provide explicit guidance on when to use this tool versus alternatives like search_transactions or list_transactions. The context is implied but not directly contrasted.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must cover behavioral traits. It does not mention that this is a read-only operation, nor any potential performance considerations, rate limits, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences: one describing the action and scope, one describing the output. No redundant information, efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no output schema and no annotations, the description is adequate but lacks usage guidelines and behavioral context. It covers the basic purpose and return fields but not when or how to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the single parameter with a clear description. The tool description adds no additional semantics beyond stating it lists accounts, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all bank accounts across configured connections and specifies the returned fields (UIDs, IBANs, names, currencies). It distinguishes from siblings like get_balance or list_transactions by focusing on account listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., get_balance for a single account). No explicit conditions or prerequisites provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_transactionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses default date range and optional filters, but does not state that the operation is read-only, nor mention pagination, rate limits, or any side effects. This is a significant gap for a tool with 8 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action and resource. Every sentence adds value: first states purpose and filters, second gives default behavior. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with 8 parameters and no output schema, the description covers defaults and filter types, but does not explain return value, pagination behavior, or typical usage scenarios. Schema descriptions fill some gaps, but overall completeness is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds little beyond the schema – it mentions 'debit/credit type filtering' which is already in the enum, and 'amount range' which is covered by 'amountMin' and 'amountMax'. No new semantic insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists bank transactions with optional filters. It is a specific verb-resource pairing. However, it does not differentiate from sibling 'search_transactions', which may have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing transactions with filters and mentions a default 90-day window, but lacks explicit guidance on when to use this tool versus alternatives like 'search_transactions' or 'spending_summary'. No exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_transactionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It describes the search operation but does not disclose whether it is read-only, any performance implications, pagination behavior, or error handling. The word 'search' implies read but is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long with no extraneous words. The first sentence states the action, the second provides usage context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters (1 required) and no output schema. While the purpose is clear, the description fails to explain optional parameters like connectionId, dateFrom, dateTo, and does not describe return format or behavior for edge cases. This leaves gaps for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 40% (only query and limit have descriptions). The description adds semantics for query (full-text across specific fields) but does not explain connectionId, dateFrom, or dateTo. With low schema coverage, the description should compensate but does not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'full-text search across transaction descriptions, merchant names, and references' with a specific use case of 'finding specific payments or payees'. This distinctly separates it from sibling tools like list_transactions which likely list all transactions without search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use the tool ('for finding specific payments or payees') but does not explicitly mention when not to use it or compare to alternatives like list_transactions. The guidance is clear but lacks explicit exclusionary context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balanceA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It does mention the return types ('closing booked balance' and 'expected balance'), which is helpful, but it does not address whether the tool is read-only, if it requires authentication (implicit via connectionId), or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, no redundant words, and directly addresses the tool's purpose and output. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple balance retrieval tool with two parameters and no output schema, the description is mostly complete. It covers the output type and the optionality of accountId. However, it could clarify terms like 'closing booked balance' and 'expected balance' for better clarity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only accountId has a description). The description for accountId adds useful context: 'If omitted, returns balances for all accounts.' However, connectionId lacks a description in both schema and tool description, leaving its meaning unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'current account balance(s)', and specifies that it returns 'closing booked balance and expected balance'. This distinguishes it from sibling tools like list_accounts or list_transactions, which deal with other account data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when to prefer get_balance over list_accounts or spending_summary, or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.