Server Quality Checklist

Tool Scores

• update_categoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an update operation (implying mutation) and shows the return format, but doesn't address important behavioral aspects like: what permissions are required, whether updates are atomic, what happens with partial updates, error conditions, or rate limits. The description adds minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by Args and Returns sections. Each sentence earns its place, though the 'Returns' statement could be slightly more specific. The formatting with sections makes it easy to parse, though it's not perfectly front-loaded (the key purpose comes first, but details follow in sections).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations but with an output schema (implied by 'Returns' statement), the description covers the basic operation and parameters well. However, for a tool that modifies financial data, it lacks important context about permissions, validation rules, error handling, and transactional behavior. The presence of an output schema reduces the need to describe return values, but behavioral context remains incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate, and it does so effectively. It explains all 5 parameters: category_id (UUID), name (new name), note (new note), goal_target (milliunits with example), and budget_id (default behavior). The goal_target example (500000 = $500.00) is particularly valuable. However, it doesn't explain null handling for optional parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool updates specific attributes of a category (name, note, goal target) with a specific verb+resource combination. It distinguishes itself from sibling tools like 'update_month_category' by focusing on category-level updates rather than month-specific category updates. However, it doesn't explicitly contrast with 'get_categories' or other update tools beyond the title difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to prefer 'update_category' over 'update_month_category' or when category updates are appropriate versus other operations. The only contextual hint is the budget_id parameter default behavior, but this isn't framed as usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_transactionB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it correctly identifies this as a destructive operation ('Delete'), it doesn't mention permission requirements, whether deletion is permanent/reversible, rate limits, or what happens to associated data. The return statement is vague ('Confirmation of deletion').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement followed by Args and Returns sections. Every sentence serves a purpose with zero wasted words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive operation with no annotations, the description is minimally adequate but incomplete. It identifies the tool as a deletion operation and documents parameters, but lacks crucial behavioral context about permissions, permanence, and side effects. The existence of an output schema somewhat mitigates the need to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by explaining both parameters: 'transaction_id' as 'The transaction UUID to delete' and 'budget_id' as 'Budget ID (uses default if omitted)'. This adds meaningful context beyond the bare schema, though it could specify format expectations for UUIDs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete') and resource ('a transaction'), making the purpose immediately understandable. However, it doesn't differentiate this tool from sibling tools like 'update_transaction' or 'get_transactions' beyond the obvious verb difference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'update_transaction' or 'create_transaction'. It doesn't mention prerequisites, consequences, or scenarios where deletion is appropriate versus modification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_payeesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states it's a list operation, implying read-only behavior, but doesn't disclose any behavioral traits such as permissions needed, rate limits, pagination, or what happens if the budget_id is invalid. This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by structured Args and Returns sections. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single optional parameter) and the presence of an output schema (which handles return values), the description is mostly complete. It covers the purpose and parameter semantics adequately, but lacks behavioral details like error handling or usage context, which are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter: it explains that 'budget_id' is optional and defaults to a default budget if omitted, which is not covered in the schema (0% coverage). This compensates well for the low schema coverage, though it doesn't detail the format or constraints of the budget_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and resource 'all payees in a budget', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_accounts' or 'get_transactions' which also list resources, though the resource type (payees) is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or compare it to other tools like 'get_transactions' that might involve payees, leaving the agent to infer usage based on context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scheduled_transactionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a list operation, implying read-only behavior, but doesn't address permissions, rate limits, pagination, or error handling. The description adds minimal context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by structured Args and Returns sections. Every sentence earns its place, with no wasted words, making it highly efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema, the description is reasonably complete. It covers the purpose, parameter semantics, and return value structure, though it lacks behavioral details like error cases or performance considerations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics for the single parameter 'budget_id', explaining it's optional and defaults to a budget if omitted. Since schema description coverage is 0%, this compensates well, though it could specify what 'default' means (e.g., active budget).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all scheduled (recurring) transactions'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_transactions', which might also retrieve transaction data but not specifically scheduled ones.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_transactions' or clarify scenarios where scheduled transactions are needed over regular ones, leaving usage context implied at best.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_monthA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's read-only nature implicitly by using 'Get' and describes the return structure, but lacks details on permissions, rate limits, error handling, or data freshness. It adds some behavioral context but not comprehensively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the purpose, followed by clear sections for Args and Returns. Every sentence adds value: the purpose statement, parameter explanations with examples, and return details. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but has an output schema), the description is fairly complete. It covers purpose, parameter semantics, and return values. However, it lacks usage guidelines and some behavioral details like error cases or dependencies, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics: 'month' is explained as 'Month in YYYY-MM-DD format (first of month)' with an example and default, and 'budget_id' is clarified with 'uses default if omitted.' This provides essential context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a monthly budget summary with category breakdowns.' It specifies the verb ('Get') and resource ('monthly budget summary'), but doesn't explicitly differentiate from sibling tools like 'get_budget_months' or 'get_budgets' that might also retrieve budget-related data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. While it mentions defaults for parameters, it doesn't specify scenarios for usage, prerequisites, or exclusions compared to siblings like 'get_budget_months' or 'get_budgets'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_transactions_bulkA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions returns a summary of created and duplicate transactions, which is helpful, but doesn't disclose critical behavioral traits like whether this is a write operation (implied by 'create'), authentication requirements, rate limits, error handling, or idempotency. The description adds some value but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose. The JSON details are necessary but could be slightly more streamlined. Most sentences earn their place, though the parameter explanations are integrated rather than separate sections.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, but an output schema exists, the description does well by detailing parameters and return summary. It covers the essential what and how, though it could improve on behavioral aspects like permissions or errors. The output schema reduces need for return value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed semantics for both parameters: 'transactions_json' is explained with a comprehensive JSON structure including required/optional fields and nested arrays, and 'budget_id' is noted as optional with a default. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates multiple transactions from a JSON string, specifying it's a bulk operation. It distinguishes from the sibling 'create_transaction' by emphasizing multiple transactions at once, though it doesn't explicitly name that alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for bulk creation of transactions, but doesn't explicitly state when to use this vs. the single 'create_transaction' tool or mention prerequisites like required permissions. It provides some context through the JSON structure but lacks explicit guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does add some useful context: it mentions server-side filtering for 'before_date,' provides a date range example, notes a default for 'max_results,' and describes the return structure including truncated flags. However, it lacks information about permissions, rate limits, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and uses bullet points effectively. Every sentence adds value, though the date range example could be slightly more concise. Overall, it's appropriately sized for an 8-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, no annotations, but with output schema), the description is quite complete. It thoroughly documents all parameters and their semantics, and while it doesn't need to explain return values (output schema exists), it usefully summarizes what's returned. The main gap is lack of behavioral context like permissions or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear explanations for all 8 parameters. It explains defaults (budget_id, max_results), date format examples, filtering logic (type options, server-side filtering), and practical usage examples like date ranges.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Search and list transactions with optional filters,' which is a specific verb+resource combination. However, it doesn't explicitly differentiate this from sibling tools like 'get_scheduled_transactions' or 'get_accounts,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_scheduled_transactions' or 'get_accounts.' It mentions that 'budget_id' uses a default if omitted, but this is parameter-specific and doesn't constitute overall usage guidance for the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_transactionA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies mutation ('Update') but doesn't specify permission requirements, whether changes are reversible, or any rate limits. The description does add some context about split transactions and parameter defaults, but lacks comprehensive behavioral details for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and uses bullet-like formatting for parameters. While comprehensive, some sentences could be more concise (e.g., the subtransactions explanation is verbose). Overall, it's efficiently organized with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (13 parameters, mutation operation, no annotations) and the presence of an output schema, the description is reasonably complete. It thoroughly documents parameters and mentions return values. However, it lacks behavioral context like error conditions or side effects that would be helpful for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic explanations for all 13 parameters. It clarifies data formats (YYYY-MM-DD, milliunits), enumerated values for 'cleared' and 'flag_color', complex requirements for 'subtransactions', and default behavior for 'budget_id'. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Update') and resource ('an existing transaction'), making the purpose immediately understandable. However, it doesn't differentiate this tool from sibling tools like 'update_category' or 'update_month_category' beyond the resource type, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_transaction' or 'delete_transaction'. It mentions no prerequisites, constraints, or typical use cases, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_accountA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states this creates a new account but doesn't disclose behavioral traits like required permissions, whether this is idempotent, error conditions, or side effects. The Returns section describes output but not behavioral context. For a mutation tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (Args, Returns), front-loaded purpose statement, and zero wasted sentences. Every element serves a purpose: the first sentence states the tool's function, followed by parameter documentation and return value description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations but with output schema (implied by Returns section), the description covers parameters thoroughly and states what's returned. However, it lacks behavioral context (permissions, errors, side effects) that would be important for a creation tool. The output schema reduces but doesn't eliminate the need for behavioral transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides clear semantic meaning for all 4 parameters: name with examples, type with complete enum values, balance with units and examples, and budget_id with default behavior. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new account') and resource ('in a budget'), distinguishing it from sibling tools like get_accounts (read) or create_transaction (different resource). The verb+resource combination is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. While it's clear this creates accounts, there's no mention of prerequisites (e.g., budget must exist), constraints, or comparison to similar operations like updating existing accounts. The description only states what it does, not when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_budgetsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool lists budgets 'the user has access to,' which implies permission-based filtering, but doesn't cover other behavioral aspects like pagination, rate limits, or error handling. The description adds some context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first, followed by parameter and return details. It avoids unnecessary fluff, though the 'Args:' and 'Returns:' sections could be integrated more seamlessly into the flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema, the description is reasonably complete. It covers the purpose, parameter effect, and return structure, though it could benefit from more behavioral context or usage guidelines to fully compensate for the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It explains the single parameter 'include_accounts' by stating 'If True, include account summaries for each budget,' adding meaningful semantics beyond the schema's title. This clarifies the parameter's effect, though it doesn't detail the format of account summaries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('budgets'), and it distinguishes from siblings by specifying 'all budgets the user has access to' rather than filtered subsets. This is precise and differentiates it from other list tools like get_accounts or get_transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, compare to sibling tools like get_budget_months, or specify scenarios where this tool is preferred over others. The usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_categoriesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It indicates this is a read operation ('List') and specifies temporal scope ('current month'), but doesn't mention permissions needed, rate limits, pagination, or error conditions. It provides basic behavioral context but lacks operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns) and uses minimal sentences. The first sentence efficiently conveys the core functionality, though the 'Args' and 'Returns' labels could be integrated more seamlessly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (read operation with one optional parameter) and the presence of an output schema (which handles return value documentation), the description is reasonably complete. It covers purpose, parameter semantics, and return content at a high level, though could benefit from more behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter: it explains that budget_id is optional ('uses default if omitted') and clarifies its purpose ('Budget ID'). With 0% schema description coverage and only one parameter, this adequately compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all category groups and categories') and specifies the exact data returned ('budgeted/activity/balance for the current month'). It distinguishes from siblings like get_accounts or get_transactions by focusing specifically on categories with financial metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving category data with monthly financial info, but doesn't explicitly state when to use this vs alternatives like get_month (which might provide similar data) or update_category (for modifications). No explicit exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_accountsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that it's a read operation ('List') and mentions balance details, but does not cover behavioral aspects like error handling, pagination, rate limits, or authentication needs. The description is adequate but lacks depth for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose statement, followed by structured 'Args' and 'Returns' sections. Every sentence adds value without redundancy, making it efficient and well-organized for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter) and the presence of an output schema (which handles return values), the description is mostly complete. It covers the purpose, parameter semantics, and return structure, but could improve by addressing behavioral transparency gaps like error cases or usage constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains that 'budget_id' is optional and uses a default if omitted, adding meaningful context beyond the schema. However, it does not detail what the default is or provide examples, leaving some gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all accounts in a budget with balances'), making the purpose specific and actionable. It distinguishes itself from siblings like 'get_budgets' or 'get_transactions' by focusing exclusively on accounts with balance information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'in a budget' and noting that 'budget_id' uses a default if omitted, providing some context. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get_budgets' or 'create_account', and does not mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_budget_monthsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool as a list operation with summary data, which implies it's read-only and non-destructive. However, it lacks details on permissions, rate limits, pagination, or error conditions. The description adds some context about trend analysis but doesn't fully compensate for the missing annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core purpose, followed by usage context, then parameter and return details. Every sentence adds value without redundancy. The bullet-point style for Args and Returns enhances readability without wasting space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (1 parameter, no annotations, but with output schema), the description is reasonably complete. It covers purpose, usage, parameters, and return fields. The output schema exists, so the description doesn't need to explain return values in detail. However, it lacks behavioral details like error handling or data freshness, which would be helpful given the absence of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It explains the single parameter (budget_id) by stating it's optional ('uses default if omitted'), which adds meaningful semantics beyond the schema's technical definition. However, it doesn't clarify what 'default' means or provide format examples, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('all months in a budget with summary data'), distinguishing it from siblings like get_month (single month) or get_budgets (budget metadata). It explicitly mentions the data fields included (income, spending, budgeting), making the scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for trend analysis — see how income, spending, and budgeting have changed over time'), which implicitly differentiates it from tools like get_month (single month detail) or get_transactions (transaction-level data). However, it does not explicitly state when NOT to use it or name specific alternatives, keeping it from a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_month_categoryA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It correctly identifies this as a mutation operation ('Update'), describes the return format, and mentions default behaviors (month defaults to current month, budget_id uses default if omitted). However, it doesn't disclose important behavioral traits like whether this requires specific permissions, if changes are reversible, rate limits, or error conditions—significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with purpose statement, usage guidance, parameter explanations, and return description—all in appropriate sections. It's front-loaded with the core purpose. While efficient, the 'Args:' and 'Returns:' sections could be slightly more concise, but overall there's minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations but with an output schema (which handles return values), the description provides good coverage. It explains the tool's purpose, usage, all parameters, and mentions what's returned. The main gap is lack of behavioral context around permissions, side effects, or error handling, which would be valuable for a budget update operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic explanations for all 4 parameters. It explains what each parameter represents (category UUID, budgeted amount in milliunits with example, month format with default, budget ID behavior), adding substantial value beyond the bare schema. The parameter documentation is comprehensive and helpful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Update the budgeted amount'), target resource ('for a category in a specific month'), and distinguishes it from sibling tools like 'update_category' (which likely updates category metadata rather than monthly budget allocations). The verb+resource combination is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Use this to move money between categories or adjust budget allocations'), which gives practical guidance. However, it doesn't explicitly state when NOT to use it or mention specific alternatives among the sibling tools (like whether 'update_category' serves a different purpose).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_transactionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a creation tool (implying mutation), specifies default values (e.g., cleared defaults to 'cleared', approved defaults to True), explains conditional logic (payee_name vs. payee_id, category_id omission with subtransactions), and details constraints (subtransactions amounts must sum to parent amount). It could improve by mentioning authentication needs or error handling, but it's quite comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and uses bullet-like formatting for readability. Most sentences earn their place by providing essential information, but it could be more front-loaded—the detailed parameter explanations are lengthy, though necessary given the 0% schema coverage. A brief summary upfront might enhance conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 parameters, no annotations, 0% schema coverage) and the presence of an output schema (implied by 'Returns' section), the description is highly complete. It thoroughly explains all input parameters, their interactions, defaults, and constraints, and summarizes the return values. This provides enough context for an agent to use the tool effectively without relying on structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It excels by providing detailed semantics for all 12 parameters: explains formats (e.g., date in YYYY-MM-DD, amount in milliunits with sign conventions), clarifies usage (e.g., payee_name creates new payee if missing, payee_id for existing), defines enums (e.g., cleared status options, flag colors), describes complex structures (subtransactions array with required fields and summation rule), and notes defaults and optionality. This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear, specific verb ('Create') and resource ('a new transaction'), immediately stating what the tool does. It distinguishes from siblings like 'create_transactions_bulk' (single vs. bulk creation), 'delete_transaction' (creation vs. deletion), and 'update_transaction' (creation vs. modification), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage through parameter explanations (e.g., 'Use this OR payee_id' for payee_name, 'Omit when using subtransactions' for category_id, 'uses default if omitted' for budget_id). However, it lacks explicit guidance on when to choose this tool over alternatives like 'create_transactions_bulk' or 'update_transaction', which would be needed for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.