Server Quality Checklist

Tool Scores

• get_categoriesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It simply states the outcome without disclosing side effects, permissions, or whether the list is exhaustive. The agent cannot infer read-only behavior or other traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no unnecessary words. It is front-loaded and effectively communicates the core function without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (0 parameters, output schema exists), the description is minimally adequate. However, it lacks context about the scope of categories (all user's? all system?), ordering, or how it relates to sibling tools, leaving some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so baseline is 4. The schema coverage is 100%, and the description adds no parameter details, which is acceptable since no parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'list of category names', making the tool's purpose evident. It distinguishes itself from sibling tools like 'get_transactions' by focusing on categories, though it does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It lists all fields returned, which adds clarity about the output. The phrase 'Get details' implicitly indicates a read-only, non-destructive operation, but an explicit statement about safety (e.g., 'This is a safe read operation') is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bullet points, but it is verbose, listing all return fields. Since an output schema exists, this redundancy reduces conciseness. The opening sentence is good, but the detailed list could be omitted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 parameter, output schema available). The description covers the parameter, explains what is returned (though redundant), and provides context. It is complete for this tool, but could explicitly state it is a read-only operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description explicitly documents the parameter as 'transaction_id: ID of the transaction to retrieve,' providing meaning beyond the schema's type-only definition. With a single parameter, this is clear and sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets details of a specific transaction, using a specific verb+resource ('Get details about a specific transaction'). It distinguishes from the sibling tool 'get_transactions' which retrieves multiple transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for a single transaction via 'specific transaction' and mentions the transaction_id parameter, but does not explicitly state when to use this tool versus alternatives like get_transactions, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_accountsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool returns a combined list with minimal information for each account, including specific fields. Since there are no annotations, it adequately covers the read-only nature and output structure, though it could mention if there are any limitations like pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences, front-loaded with the main action, and each sentence adds meaningful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description adequately covers the tool's purpose and return value. It lists the main fields but could mention error conditions or prerequisites for completeness, though the simplicity of the tool mitigates this need.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the baseline score for such cases is 4. The description adds value by detailing the types of accounts included and output fields, which goes beyond what the empty schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get all accounts' and specifies it includes both manual and Plaid-synced accounts, distinguishing it from sibling tools that handle other resources like categories and transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Useful for understanding which accounts are available and their current balances,' which implies a usage context, but it does not explicitly state when to use this tool over alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_current_userA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It implies a read-only operation by stating it returns user information, but it does not explicitly confirm no side effects or mention authentication requirements. The listed return fields add some transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: first states the core function, second lists key fields and use cases. No unnecessary words; front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and an output schema (presence noted). The description covers the return fields and usage, and sibling tools are distinct. Complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters and schema coverage is 100%, so the description's role is minimal. It adds value by listing return fields, though this pertains to output rather than parameters. Baseline 4 applies for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves details about the current Lunch Money user, listing specific fields like name, email, user_id. It distinguishes itself from sibling tools (e.g., get_categories, get_transactions) which target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes the tool is 'useful for verifying authentication and understanding the account context,' providing clear usage context. It does not explicitly exclude alternatives, but the context is sufficient for an agent to decide when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_numbersA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description details return format (rounded to 2 decimal places) and supports negative numbers for subtraction. Sufficiently transparent for a simple addition tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear Args and Returns sections. Every sentence is useful, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input schema and output schema mentioned, the description fully covers behavior, parameter usage, and return format. Complete for the tool's purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description explains the numbers parameter as 'List of numbers to add together' and mentions negative values for subtraction, adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Helper tool for adding numbers together' and provides concrete use cases like summing expenses and totals, distinguishing it from unrelated sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using this tool for arithmetic operations to avoid errors, with examples. Does not explicitly state when not to use, but context makes it clear it's for arithmetic only.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description thoroughly discloses pagination, return structure (transactions with category names), and aggregate behavior. It provides all necessary behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear purpose, pagination warning, parameter list, and return details. Slightly verbose but every sentence is informative. Front-loaded with essential info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully covers the tool's complexity: 13 params, pagination, aggregates, return format. With no annotations and an output schema present, the description is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description explains all 13 parameters with formats, defaults, and dependencies (e.g., 'include_pending ignored if is_pending set'). Adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get transactions for a date range' and details the functionality, distinguishing it from sibling tool 'get_transaction'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs about pagination, how to handle 'has_more', and when to make additional requests. Also clarifies 'include_aggregates' behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.