Server Quality Checklist

Tool Scores

• get_net_worthA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosure. It does not mention whether results are real-time or cached, or if special authentication is needed. The description only specifies the output content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that immediately states the tool's purpose and output structure. No superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and the existence of an output schema, the description provides a complete picture of what the tool returns: a net worth summary with breakdowns. No additional context is necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description cannot add meaning beyond the schema. Baseline of 4 is appropriate as the description is not required to elaborate on nonexistent parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets a summary of current net worth with a breakdown into assets vs liabilities and by account category. It effectively differentiates from siblings like get_asset_allocation and list_accounts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description implies it's for a high-level overview, but lacks explicit when or when-not recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_asset_allocationA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It explains that the tool shows allocation by asset class with dollar values and percentages, and lists holdings per account. However, it does not mention whether data is real-time or historical, any authentication requirements, or if the tool is read-only (assumed safe). Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences: the first states the main action, the second elaborates on output details, and the third provides parameter usage. It is front-loaded with the core purpose, with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (context signals indicate true), the description does not need to explain return structure. It sufficiently covers the key behaviors (asset class breakdown and per-account listing), includes parameter usage, and matches the tool's complexity. Minor omission: no mention of data scope (e.g., all accounts or filtered).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the only parameter, account_filter. The description adds value by stating 'Use account_filter to focus on retirement accounts, a specific brokerage, etc.,' which gives contextual usage beyond the parameter name. This compensates for the schema's lack of descriptions, though format constraints are not detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Show investment holdings and asset allocation breakdown,' clearly specifying the verb and resource. It further details the output format (by asset class with values and percentages, then per account), making the tool's purpose unambiguous and distinguishing it from siblings like get_transactions or get_net_worth.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives. It provides parameter guidance ('Use account_filter to focus on retirement accounts...') but lacks comparison or exclusions relative to sibling tools. The purpose implies usage context, but direct guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses key behavioral traits: returns up to 100 transactions sorted most-recent first. This is good context for an agent. Missing details like authentication needs or side effects, but given it's a fetch operation, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, filtering, output characteristics. No redundancy and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema (not shown), the description does not need to detail return values. It covers purpose, filtering, and output limits. However, the unexplained 'days' parameter is a gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 3 parameters: days, search, min_amount. Description explains search (keyword, merchant/description) and min_amount (dollar amount) but omits any explanation of the 'days' parameter. Schema description coverage is 0%, so the description should compensate fully but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Fetch' and resource 'transactions' with scope 'from all connected accounts'. This distinguishes it from siblings like list_accounts (accounts) and get_net_worth (summary) which are clearly different.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description mentions filtering capabilities (keyword, min_amount) but does not explicitly state when to use this tool versus alternatives or provide when-not-to-use scenarios. However, sibling tool names make the differentiation obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_auth_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses it returns a status message and provides fix instructions. No contradictory annotations. For a read-only status check, this is adequate, though it doesn't detail potential side effects or session refresh behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences that front-load the main purpose. Every sentence adds value with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully adequate given no parameters and presence of an output schema. Covers purpose, behavior, and guidance for an auth check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and schema coverage is 100%. Description adds no param-specific info, which is acceptable as there are none to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool checks authentication status for a Personal Capital session. The verb 'check' and resource 'auth status' are specific and distinguish from sibling tools that retrieve financial data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies use before other operations by stating it checks validity and offers guidance if not authenticated. However, it does not explicitly state when to use or not use it, nor mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing that closed accounts are excluded, zero-balance accounts are shown by default, and grouping by type. It omits authentication requirements and error handling, but covers core behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each serving a distinct purpose: core function, grouping, default behavior, and filtering option. No extraneous words, well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, the description need not explain return values. It covers key behaviors (grouping, closed account exclusion, zero-balance handling, filtering). Missing authentication or error context, but sufficient for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must add meaning. It explains the purpose of both parameters: type_filter narrows to a category with enum values mapped to account groupings, and hide_zero_balance controls whether zero-balance accounts are hidden. This adds significant value beyond the schema's bare enum and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all Personal Capital / Empower accounts with current balances', specifying the verb (list), resource (accounts), and scope (all, with balances). It differentiates from siblings by detailing grouping by type and behaviors like excluding closed accounts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (to list accounts) and mentions the type_filter for narrowing categories. However, it does not explicitly state when not to use it or offer alternative tools; sibling tools like get_net_worth or get_transactions imply different use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.