Server Quality Checklist

Tool Scores

• get_net_worthA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It only describes the output structure but does not mention whether data is real-time, cached, requires authentication, or if any side effects occur. A read-only hint or access requirements are missing, leaving behavioral expectations unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the tool's purpose and output breakdown. While clear and concise, it could be slightly more structured (e.g., bullet format) but is generally effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an existing output schema, the description adequately covers the tool's return structure (assets vs liabilities, category subtotals). It implicitly defines net worth but does not note whether data reflects live or historical state. Overall, it is sufficiently complete for a simple summary tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters in the input schema, and schema description coverage is 100%. Per the calibration rule, 0 parameters warrant a baseline of 4. The description does not need to add parameter semantics as none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a summary of current net worth, broken into total assets vs total liabilities with subtotals by category. This specific verb+resource combination distinguishes it from sibling tools like get_asset_allocation (allocation breakdown) and list_accounts (account listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For example, it does not mention that get_asset_allocation is more appropriate for investment-specific breakdowns or that get_transactions provides granular ledger data. The description simply states what it does without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_asset_allocationA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses output structure ('allocation by asset class... dollar values and percentages, then lists holdings per account') but omits behavioral details like rate limits, data freshness, or that it's read-only. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each adding value: first states purpose, second elaborates on output, third guides parameter use. No redundant words, well front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 optional param, output schema exists), the description covers tool function and filter usage effectively. It could improve by mentioning default behavior (all accounts) or edge cases, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, account_filter, has no schema description (0% coverage). The description compensates by explaining its purpose and providing usage examples ('focus on retirement accounts, a specific brokerage, etc.'), adding meaningful guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Show investment holdings and asset allocation breakdown' with specific verb 'show' and resource. It distinguishes from siblings like get_net_worth, get_transactions, list_accounts, and check_auth_status, which cover different financial data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for the account_filter parameter with examples ('retirement accounts, a specific brokerage, etc.'), guiding when to use it. However, it does not explicitly state when not to use this tool or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_auth_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool returns a status message and offers guidance if unauthenticated. However, it does not mention whether the check has side effects or any specific behavior beyond the stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, contains no fluff, and is front-loaded with the essential purpose. Every sentence provides value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description adequately covers the purpose and behavioral hints. It is complete for a simple auth check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter info, but none is needed due to the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it checks whether the session is authenticated and valid, using a specific verb and resource. It distinguishes from sibling tools, which focus on financial data rather than auth status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for authentication status checking but provides no explicit guidance on when or when not to use it, nor does it mention alternatives. Sibling tools are clearly different in purpose, but no direct comparison is made.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the disclosure 'Returns up to 100 transactions sorted most-recent first' provides important behavioral traits (limit and sorting). It does not mention rate limits, authentication requirements, or side effects, but for a read operation it is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences that front-load the main purpose and efficiently convey all key information. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 optional params, output schema exists), the description covers purpose, filtering, limit, and sort order. It does not mention the default for 'days' or confirm that it returns only transactions from connected accounts (stated in purpose). Mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the schema by explaining 'search' filters by keyword (merchant/description) and 'min_amount' sets a minimum dollar amount. The 'days' parameter is implied by 'recent' but not explicitly described, leaving a minor gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch recent transactions from all connected accounts,' specifying the verb (fetch), resource (transactions), and scope (all accounts). This differentiates it from siblings like 'list_accounts' or 'get_asset_allocation'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides when-to-use guidance through filtering options ('keyword (merchant/description)' and 'minimum dollar amount'). It does not explicitly state when not to use or name alternatives, but the purpose is distinct enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description details grouping by type, exclusion of closed accounts, and default inclusion of zero-balance accounts. It implies read-only behavior but lacks explicit mention of side effects or data freshness. Output schema exists, reducing the need for return value details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences, each adding value, with the purpose front-loaded. No wasted words, efficiently conveys all key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two parameters and an output schema, the description covers all essential aspects: what the tool does, defaults, filtering, and grouping. No gaps remain given the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description compensates for 0% schema coverage by explaining the purpose of type_filter (narrow to category) and hide_zero_balance (overrides default shown behavior), providing clear semantics beyond the schema enums and default values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists accounts with current balances, specifying the resource (accounts), verb (list), and context (Personal Capital / Empower). It distinguishes from sibling tools like get_transactions or get_net_worth.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains default behavior (zero-balance shown, closed excluded) and the type_filter parameter, but does not explicitly state when to use this tool versus alternatives or provide conditions for not using it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.