Server Quality Checklist

Tool Scores

• get-recent-transactionsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Get recent transactions' implies a read operation but reveals nothing about permissions, rate limits, pagination, error conditions, or what 'recent' means contextually. For a tool with zero annotation coverage, this leaves critical behavioral traits undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at three words with zero wasted text. It's front-loaded and efficiently communicates the core function without unnecessary elaboration. While under-specified, it earns full marks for brevity and structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is incomplete. It fails to explain what 'recent' means, how results are ordered, what data fields are returned, or how it differs from sibling tools. The agent lacks sufficient context to use this tool effectively without trial and error.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('days' and 'limit') well-documented in the schema. The description adds no parameter semantics beyond what the schema already provides. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get recent transactions' is a tautology that essentially restates the tool name without adding meaningful differentiation. It specifies the verb 'Get' and resource 'transactions' but lacks specificity about scope or how it differs from sibling tools like 'search-transactions'. This provides minimal value beyond the name itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search-transactions' or 'get-category-spending'. There's no mention of context, prerequisites, or exclusions. The agent must infer usage from the name alone, which is insufficient for informed tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-budget-summaryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves a 'budget summary' but doesn't explain what that summary includes (e.g., categories, totals, trends), whether it's read-only (implied but not explicit), or any limitations like rate limits or authentication needs. This leaves significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the 'budget summary' output contains, how it's structured, or any behavioral traits like error handling. For a tool with two parameters and no structured output documentation, this leaves the agent with insufficient context to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters ('start_date' and 'end_date') fully documented in the schema. The description adds minimal value by mentioning 'time period,' which aligns with the schema but doesn't provide additional context like format examples or edge cases beyond what's already specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('budget summary') with a specific scope ('for a specific time period'), which makes the purpose understandable. However, it doesn't differentiate this tool from its siblings like 'get-category-spending' or 'get-recent-transactions', which likely provide related but different budget/transaction data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get-category-spending' or 'get-recent-transactions', nor does it specify prerequisites, exclusions, or appropriate contexts for usage beyond the time period requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-category-spendingC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get spending' implies a read operation, but it doesn't specify if this requires authentication, has rate limits, returns historical or real-time data, or what format the spending data is in. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with two parameters) and the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'spending' entails (e.g., total amount, list of transactions), how results are returned, or any behavioral traits, leaving the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't add any parameter semantics beyond what the schema provides. With 100% schema description coverage, the schema already documents 'category' as 'Category name' and 'days' as 'Number of days to look back' with a default of 30. The baseline score of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get spending in a category' clearly states the verb 'Get' and resource 'spending in a category', making the purpose understandable. However, it doesn't distinguish this tool from siblings like 'get-budget-summary' or 'get-recent-transactions' which might also involve spending data, so it lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention siblings like 'search-transactions' for broader queries or 'get-budget-summary' for aggregated data, leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-transactionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'search' implies a read operation, the description doesn't address important behavioral aspects like whether this requires authentication, what happens with no results, whether results are paginated, or any rate limits. For a search tool with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three words, with zero wasted language. It's front-loaded with the essential information (search transactions) and specifies the mechanism (by keyword) efficiently. Every word serves a clear purpose in this minimal description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 3 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what constitutes a 'transaction' in this context, what fields are searched, the format of results, or how the search algorithm works. The agent would need to guess about the tool's behavior and output based on minimal information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions searching 'by keyword' which aligns with one of the three parameters. However, with 100% schema description coverage, all parameters are already well-documented in the schema itself. The description adds minimal value beyond what's in the structured schema, meeting the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('search') and resource ('transactions'), and specifies the search mechanism ('by keyword'). However, it doesn't differentiate this tool from its sibling 'get-recent-transactions' which also deals with transactions, leaving some ambiguity about when to use one versus the other.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention the sibling tools (get-budget-summary, get-category-spending, get-recent-transactions) or explain when keyword searching is preferable to other transaction retrieval methods. The agent receives no contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.