Server Quality Checklist

Tool Scores

• balance-historyC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only says the tool gets balance history. It does not explain whether the result is a time series, aggregated points, or whether includeOffBudget changes what is returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no fluff or repetition. It is appropriately brief, though it achieves conciseness at the cost of useful detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and three required parameters, a one-line description is incomplete. It leaves the agent uncertain about the return format, how months are interpreted, and what includeOffBudget controls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description adds no meaning for accountId, includeOffBudget, or months. The agent must infer their roles from names alone, and the description does not compensate for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear operation on a specific resource: getting account balance history over time. It does not explicitly distinguish itself from siblings like get-transactions or monthly-summary, but the resource and temporal scope are reasonably identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. There is no mention of scenarios, exclusions, or why an agent should prefer balance-history over get-transactions or monthly-summary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-transactionsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. 'Get' implies a read operation, but the description omits pagination, sorting, date formats, amount semantics, and any caveats about response behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler, and the core operation is front-loaded. However, it is slightly under-specified for a tool with 8 parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, output schema, and parameter descriptions, this one-liner is insufficient for an agent to use the tool reliably. An agent could make a basic call with accountId, but the filtering parameters and their formats remain unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the 8 parameters. It only maps 'account' to accountId and broadly signals that other parameters are optional filters; it does not explain limit, date ranges, amount bounds, payeeName, or categoryName.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('transactions for an account'), making the core purpose clear. It is distinct from sibling tools like monthly-summary or spending-by-category, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to choose this tool over the sibling aggregation tools, nor any mention of prerequisites, exclusions, or typical use cases. 'Optional filtering' gives a hint but not enough to route an agent confidently.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• monthly-summaryC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get' implies a read-only operation, and the description lists output categories, but it does not explain how the monthly window is calculated, what 'savings' includes, or whether accountId scopes the data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no filler or redundancy. The key output areas are front-loaded and every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and 0% parameter description coverage, the overall context is too thin. An agent cannot tell from the description how the optional accountId interacts with the summary, what time range is covered, or what the response structure looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to compensate, but it never mentions the 'months' parameter or the optional 'accountId' parameter. 'Monthly' weakly hints at the time dimension, but the default value and account scoping behavior are left unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Get'), resource ('monthly summary'), and the key data areas ('income, expenses, and savings'), making the tool's core function clear. It does not explicitly contrast with siblings like spending-by-category or balance-history, but the summary-style output is reasonably distinguishable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to prefer this tool over alternatives such as get-transactions, spending-by-category, or balance-history. Usage is only weakly implied by the phrase 'monthly income, expenses, and savings' rather than explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spending-by-categoryC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. The word 'Get' implies a read-only operation, and 'breakdown by category' communicates aggregation behavior. However, it does not disclose date inclusivity, whether income is treated differently, category rollup behavior, or what the response contains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no redundant wording. It is concise, though the conciseness comes at the cost of omitting important usage and parameter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given four undocumented parameters, no annotations, and no output schema, the description is too thin to fully prepare an agent. An agent can infer the basic intent but cannot confidently know how accountId or includeIncome affect results, what date formats are expected, or what the return structure looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'date range', which partially maps to startDate and endDate, but it provides no meaning for accountId or includeIncome, and no format or default information for any parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Get spending breakdown by category') and a clear resource and scope ('specified date range'). It does not explicitly differentiate from sibling tools like 'get-grouped-categories' or 'monthly-summary', so it is clear but not fully distinguishing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. It does not mention when to prefer 'get-grouped-categories' or 'monthly-summary', nor does it explain any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-budget-monthA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It uses 'Retrieve' to indicate a read operation and lists the data included. It does not mention response format, pagination, or authentication, but these are less critical for a simple month-scoped lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the verb and resource, and every word adds value. 'Budgeted amounts, spending, and balances per category' is precise and free of filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter, full schema coverage, and no output schema, the description adequately explains what is returned. It misses only a note on how this differs from overlapping siblings, which is covered by the usage dimension.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the only parameter 'month' is already described with format and example. The description adds no extra parameter semantics beyond confirming the month scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves budget data for a specific month with category-level budgeted amounts, spending, and balances. It is specific about the resource and scope, though it does not explicitly differentiate from siblings like monthly-summary or get-budget-months.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the description: to get monthly budget data per category. However, there is no guidance on when to choose this over similar siblings such as monthly-summary, spending-by-category, or get-budget-months.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-budget-monthsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It conveys that the operation is a read-only retrieval and discloses the output format, but does not mention ordering, empty results, or how this differs from the singular get-budget-month tool. Adequate for a simple getter, but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that directly states the action and the output format with no filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list retrieval tool, the description fully specifies what is returned and in what format. It could note the relationship to get-budget-month or clarify edge cases like empty lists, but the basic contract is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty with 100% coverage. The description's mention of YYYY-MM is output format rather than parameter semantics, which is appropriate. Baseline 4 applies for a no-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Retrieve') and resource ('all available budget months'), and specifies the output format (YYYY-MM). It does not explicitly differentiate from the sibling get-budget-month, though the plural 'months' and 'list of all' imply a distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used when a caller needs to enumerate available budget months, but it does not explicitly state when to use it versus alternatives like get-budget-month or monthly-summary. No exclusions or alternative routing are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-grouped-categoriesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'Retrieve a list' clearly indicates a read-only operation, which is useful. However, it does not disclose ordering, filtering, pagination, permissions, or any potential limitations, leaving some behavioral gaps for a tool without annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is a single focused sentence that immediately states the action, target resource, and output contents. There is no redundant wording or filler, making it easy to parse and act on.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-argument retrieval tool, the description covers the main need: what data will be returned. It names id, name, type, and category list. However, without an output schema, it does not clarify the structure of the 'category list' or whether results are sorted, so it falls just short of fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema fully documents this with 'This tool does not accept any arguments.' The description adds no parameter-specific detail, but none is needed; the baseline for a zero-parameter tool is appropriately high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Retrieve') and identifies the exact resource ('all category groups') plus the returned fields. This clearly differentiates the tool from siblings like get-payees, get-accounts, and get-budget-months, leaving no ambiguity about what it does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when the caller needs category group metadata, but it does not explicitly state when to prefer this tool over alternatives or mention any exclusion criteria. The resource name and contents make the intended use reasonably clear, but no direct guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-payeesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The verb 'Retrieve' clearly indicates a read-only operation and 'all payees' clarifies that results are unfiltered. However, because no annotations are provided, the description carries the full behavioral burden; it does not disclose result ordering, pagination, nullability of fields, or how errors are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence states the action, resource, scope, and returned fields with no filler. The key information is front-loaded and every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only listing tool with no output schema, the description is largely complete: it identifies the resource, scope, and exact return fields. Minor omissions such as sort order and response format prevent a perfect score, but they are unlikely to block correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters and the input schema explicitly states 'This tool does not accept any arguments.' With 100% schema coverage, the description does not need to add parameter detail; the baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Retrieve') with a concrete resource ('all payees') and enumerates the returned fields (id, name, categoryId, transferAccountId). The resource is unique among the sibling tools, so there is no ambiguity about what this tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a complete payee list is needed, and no sibling tool is payee-specific, but it does not explicitly state when to prefer this over alternatives or any exclusion criteria. Usage context must be inferred from the tool name and resource rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-rulesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure burden. It does this by stating the operation is a retrieval and by documenting a non-obvious output detail: amounts are in cents and signed as positive for deposits and negative for payments. It does not cover pagination, ordering, or output structure, but for a parameterless list tool these are acceptable gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The primary purpose is front-loaded, and the second sentence adds essential unit and sign semantics that are directly relevant to interpreting the result.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-argument tool with no output schema, the description provides the essential purpose and one important output convention, but it leaves some ambiguity about what fields a rule contains and how the list is ordered. Still, an agent can confidently select and invoke the tool based on this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema declares zero parameters and explicitly states the tool accepts no arguments, so parameter-level documentation is unnecessary. With schema description coverage at 100% and no parameters to explain, the baseline of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Retrieve a list of all rules.' This clearly identifies what the tool does and distinguishes it from siblings like get-transactions or get-payees by naming a unique resource. The additional amount convention reinforces that the tool has a defined output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used when an agent needs all rules, but it does not explicitly state when to choose it over alternatives or mention any exclusions. The wording gives a clear purpose, but no direct comparison to sibling tools or conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-accountsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It clearly communicates a read-only retrieval operation and the returned content. It doesn't mention pagination, ordering, or exact response shape, but for a zero-parameter list tool the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence that front-loads the verb and resource, names the returned fields, and contains no filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless retrieval tool, the description covers the essential information: what is returned and its scope ('all accounts'). The lack of output schema is partially compensated by naming balance and ID as the returned fields, though details like response envelope or ordering are not stated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides no parameter documentation to rely on. The baseline for zero-parameter tools is 4, and the description appropriately focuses on what is returned rather than on parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve') and a precise resource ('a list of all accounts') with clear return fields ('current balance and ID'). This makes it immediately distinguishable from sibling tools that deal with transactions, categories, budgets, and payees.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: when all accounts with their current balance and ID are needed. It does not explicitly mention alternatives or exclusions, but the domain (accounts) is distinct enough from the siblings that routing is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.