Server Quality Checklist

Tool Scores

• upload_invoiceA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided. The description does not disclose behavioral traits such as idempotency, file size limits, authentication requirements, or what 'processing' entails. For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence with no waste. It could add a sentence about return values or processing behavior, but remains appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 parameter, no output schema), the description adequately states what the tool does. However, it omits details about the processing result, error handling, and post-upload behavior, leaving gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to the sole parameter 'file_path' beyond the schema by specifying it is a local receipt file and listing accepted formats (PDF/PNG/JPEG/WebP). With 0% schema coverage, this compensates well but could mention path format requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Upload', resource 'local receipt file', and destination 'to Fintaro for processing'. It lists accepted file formats (PDF/PNG/JPEG/WebP), clearly distinguishing from sibling tools like get_invoice and list_invoices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a receipt file is available for upload, but provides no explicit guidance on when not to use it or alternatives. No exclusions or context for choosing this over other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_invoiceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides basic behavioral info (read-only, PII-free) but lacks details on authentication, error handling, or side effects. It adds value but could be more comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action ('Return') and contains no extraneous words, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and no annotations, the description does not explain what fields are in the summary or handle error cases, leaving the agent with incomplete understanding of the tool's full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explicitly explain the invoice_id parameter beyond implying it identifies the invoice. The parameter's role is inferred but not detailed, which is insufficient for full clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a summary of a single invoice, specifying it is narrow and PII-free, which distinguishes it from siblings like list_invoices (multiple invoices) and upload_invoice (create).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when needing a summary of one invoice, but does not explicitly state when not to use it or mention alternatives like list_invoices for broader queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• whoamiA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the basic function but lacks details on side effects, authentication, or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence, under 20 words, that is front-loaded and contains no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description is complete enough. It fully explains the tool's purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4. The description adds meaning by specifying what information is shown (organization and scopes), which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Show') and resource ('organization and scopes'), and it distinguishes itself from sibling tools that deal with invoices and transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when or when not to use this tool. It simply states what it does, without context or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_invoicesA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description bears full burden. It discloses pagination behavior (hasMore, offset/limit clamping, unknown total), ordering (newest-first), and return fields. It also mentions PII-free summaries, hinting at data safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (3 sentences) and front-loaded with the purpose. No unnecessary words, and it efficiently covers ordering, pagination, and return fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool, it covers pagination, ordering, and return fields. No output schema exists, so the return field list helps. Missing details include error behavior and exact invoice summary structure, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage. The description explains pagination using limit and offset but does not define offset precisely (e.g., record index). It compensates partially by showing usage pattern 'advance offset by limit'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it lists a page of invoices as narrow, PII-free summaries. The verb 'list' and resource 'invoices' are clear. Sibling tools like get_invoice (single invoice detail) and list_transactions (different resource) indicate differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear pagination instructions (offset/limit, hasMore clamping) and ordering (newest-first). It does not explicitly state when to avoid this tool (e.g., for detailed invoices use get_invoice), but the context of 'narrow summaries' implies that.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_transactionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses pagination behavior, limit clamping to 200, and date filtering. Does not explicitly state safety, but listing is read-only by nature. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences covering purpose, pagination, and filtering, each earning its place. Front-loaded and no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, paging, filtering, and return fields despite no output schema. Lacks permissions or rate limits, but acceptable given no annotations. Complete enough for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description compensates by explaining limit clamping, offset advancement, and ISO date examples with filtering purpose. Adds meaning beyond schema defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists one page of bank transactions as narrow summaries, with a specific verb and resource. It distinguishes itself from sibling tools like list_invoices and list_unmatched.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear pagination guidance (hasMore flag, offset/limit mechanics) and date filtering to avoid paging all history. Lacks explicit when-not-to-use, but siblings are distinct enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_unmatchedA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses paging behavior, response fields, and a use case. However, it could be more thorough about idempotency or side effects, though list operations are inherently safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose. Each sentence serves a distinct function: purpose, contract explanation, example. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters and no output schema, the description covers paging and response fields. It lacks explicit details about the 'transactions' array items, but the name and context are sufficient for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% parameter descriptions, but the description explains the paging contract for limit, offset, date_from, date_to, and how to use them in relation to list_transactions. This adds significant meaning beyond the schema's bare types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List one page of transactions that still need a matching receipt.', which is a specific verb+resource combination. It distinguishes from siblings like list_transactions and list_invoices by focusing on unmatched items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly references the paging contract of sibling list_transactions and provides a concrete example (month-end close). While it doesn't state when not to use, the context is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.