Server Quality Checklist

Tool Scores

• get_item_historyA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the return format in detail (list of purchases with specific fields and aggregated statistics), which is valuable context. However, it lacks information on potential limitations, such as data availability, error handling, or performance considerations (e.g., large datasets). This partial coverage results in a baseline score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, starting with the core purpose, detailing return values, and listing parameters. Each sentence serves a clear purpose: the first states the action, the next two outline outputs, and the last specifies args and returns. It could be slightly more concise by avoiding redundancy with the schema, but overall, it's efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, read-only query), no annotations, and the presence of an output schema (implied by 'Returns' section), the description is reasonably complete. It explains what the tool does, what it returns, and the parameters, compensating for the lack of annotations. However, it misses some behavioral context like data source or limitations, preventing a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning the input schema already fully documents both parameters ('item_type' and 'time_range_days') with descriptions and defaults. The description repeats some of this information (e.g., examples for 'item_type' and the default for 'time_range_days'), adding minimal value beyond the schema. According to the rules, with high schema coverage, the baseline is 3 even without additional param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Query purchase history for a specific item type,' which is a specific verb ('query') + resource ('purchase history') combination. It distinguishes itself from sibling tools like 'import_receipt_from_pdf' (data ingestion) and 'list_item_types' (metadata listing) by focusing on historical data retrieval. However, it doesn't explicitly contrast with potential alternatives for querying history, keeping it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving purchase history based on item type and time range, but provides no explicit guidance on when to use this tool versus alternatives. There's no mention of prerequisites, such as needing existing data from 'import_receipt_from_pdf,' or exclusions like not handling real-time data. This leaves usage context somewhat vague, relying on the agent to infer from the purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• import_receipt_from_pdfA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by detailing the multi-step behavior (extraction, parsing, categorization, storage). It explains the hybrid categorization approach and mentions database storage, though it could add more about error handling, performance, or permissions needed for file access.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening sentence, bullet points for steps, and separate sections for Args and Returns. It's appropriately sized but could be slightly more concise by integrating the Args section into the main text since it repeats schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-step processing), no annotations, and the presence of an output schema, the description is fairly complete. It outlines the process and return values, though it could benefit from mentioning error cases or limitations (e.g., PDF quality requirements).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'pdf_path'. The description adds minimal value beyond the schema by restating the parameter in the 'Args' section without providing additional context like file format constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Import and parse a receipt from a PDF file') and details the multi-step process (extract text, parse metadata, extract line items, categorize, store). It distinguishes itself from sibling tools like 'get_item_history' and 'list_item_types' by focusing on data ingestion rather than querying existing data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to process PDF receipts, but lacks explicit guidance on when to use this tool versus alternatives (none mentioned) or any prerequisites. It doesn't specify scenarios where this tool is preferred or when it should be avoided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_item_typesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior by specifying what statistics are returned (total purchases, recent purchase date, total amount spent) and the output format (dictionary with list). It does not cover aspects like rate limits or error handling, but provides solid operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by bullet points for statistics and a usage note. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters), no annotations, and the presence of an output schema (which handles return values), the description is complete. It covers purpose, behavior, usage context, and output semantics adequately without needing to explain parameters or duplicate schema details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description adds no parameter information, which is appropriate given no parameters exist, and it does not mislead about inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all item types/categories in the database'), and distinguishes it from siblings by specifying it returns statistics rather than individual item history or PDF imports. It goes beyond the name/title to explain what 'list' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for getting an overview of all tracked expense categories'), which implicitly differentiates it from siblings focused on history or imports. However, it does not explicitly state when not to use it or name alternatives, keeping it from a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.