Server Quality Checklist

Tool Scores

• parse_invoiceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Documents cost ($0.02 per document, x402 USDC on Base) and file size limit (20MB). Missing rate limits, idempotency, error behavior, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose and returns, followed by supported formats and parameter list. Efficiently written, but cost info placed at end and could be integrated earlier.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input requirements, parameters, cost, and supported formats. Lacks output schema description, error handling, timeout, and authentication details. Adequate but missing operational context given no output schema and expected complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning to all 4 parameters with details not in schema: file_url (public URL, max 20MB), language (document language, default ita+eng), extract_line_items (boolean, default true), validate_totals (boolean, default false). Fully compensates for 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Extract structured data from an invoice PDF' and specifies return fields (seller, buyer, line items, totals, payment info). Mentions support for European and international formats, differentiating from sibling tools like parse_bank_statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for invoices via tool name and supported formats, but provides no explicit guidance on when to use or when not to use, nor mentions alternatives like parse_generic_document.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_bank_statementA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses cost ($0.03 per document via x402 USDC on Base), file size limit (20MB), default language (ita+eng), and auto-categorization option. It does not detail error behavior or idempotency, but these are less critical for a read-only parse tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, starting with the purpose, then return type, followed by a bullet list of arguments with defaults. Every sentence adds value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and no annotations, the description covers purpose, return structure, constraints (file size, cost, defaults). It lacks error handling or format details but is sufficient for a straightforward parse tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description compensates well. For file_url it adds max file size; for language it explains OCR usage and default; for categorize_transactions it explains auto-categorization and default. This adds meaningful context beyond schema titles and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it extracts structured data from a bank statement PDF, listing specific return fields (transactions, balances, account holder info, fees). It distinguishes from sibling tools like parse_contract, parse_generic_document, parse_invoice, and supported_document_types by focusing solely on bank statements.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for bank statement PDFs but does not explicitly state when to use this tool versus alternatives (e.g., parse_invoice for invoices). No 'when not to use' or explicit context is provided, though sibling names help differentiate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• supported_document_typesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description correctly identifies the tool as free (no cost). For a read-only listing with zero side effects, this is sufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The first sentence states the purpose, the second adds a key behavioral note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters and no output schema, the description covers the essentials: what it does and that it's free. It could mention the return format, but the context is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description does not need to add parameter information. The baseline for zero parameters is 4, and the description is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all supported document types with their descriptions and prices', which is a specific verb+resource combination. It differentiates from sibling parse tools by being a listing instead of parsing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly notes the tool is 'always free to call', implying no usage restrictions. While it doesn't explicitly state when to use vs. alternatives, the context of sibling parse tools suggests this is for discovery before parsing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_contractA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses important behavioral traits: cost ($0.05 via x402 USDC on Base), file size limit (max 20MB), and default languages. It does not cover failure modes or auth needs, but provides good context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear 'Args' section, each sentence adds value. It is concise yet complete, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers purpose, parameters, cost, and output structure (JSON with specific fields). Lacks error handling details but is adequate for a 4-param tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by adding meaning to each parameter: file_url (public URL, max 20MB), language (default ita+eng), extract_clauses and extract_financial_terms (defaults and what they extract).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Extract structured data') and target ('contract PDF'), differentiating it from siblings like parse_invoice or parse_bank_statement by specifying contract-specific extraction fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides use by listing extracted content (parties, dates, financial terms, clauses), making it clear for contracts. However, it lacks explicit 'when to use' or 'alternatives' guidance, leaving room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_generic_documentA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses cost ($0.01 per doc via USDC), file size limit (20MB), default language (ita+eng), and that it extracts text and tables. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences plus an arg list, front-loaded with purpose, every sentence adds value. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 0% schema coverage, the description covers purpose, usage, parameters, cost, and limits. Returns description is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaning for all three parameters: file_url (public URL, max 20MB), language (OCR languages, default ita+eng), extract_tables (boolean, default true).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it extracts text and tables from PDF documents. It positions itself as less structured than specialized parsers (siblings) but more flexible, providing clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to use this tool (generic PDFs, when flexibility is needed) versus specialized parsers. It also mentions the cost, aiding decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.