Server Quality Checklist

Tool Scores

• get_conversionC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It indicates a read-like operation (checking status) but does not explicitly state it is non-destructive, lacks side effects, or requires any permissions. The description is minimal and does not cover idempotency or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a single sentence that front-loads the status values. However, it could be structured to include parameter guidance or usage notes without adding length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status-check tool with one parameter and no output schema, the description is adequate but incomplete. It misses guidance on the relationship to get_conversion_result and does not explain the parameter. The lack of annotations and schema descriptions increases the burden on the description, which it only partially meets.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, conversion_id, is listed in the schema but not described in the tool description, and the schema has no description. The description adds no meaning beyond the parameter name, leaving the agent to infer its format or source (e.g., from list_conversions).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks a conversion's status and lists the possible statuses (queued, processing, completed, failed). It also mentions the additional per-account summary when completed. However, it does not explicitly differentiate from sibling tools like get_conversion_result, which may have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_conversion_result or list_conversions. There is no mention of prerequisites, context, or exclusions, leaving the agent without decision support for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_conversion_resultA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description adds context on format and provenance options, but does not disclose read-only nature, authentication needs, or response structure (e.g., list/object, size limits).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-loading the main purpose, followed by details on format and provenance. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and 4 parameters (only 1 described in schema), the description adds format and provenance details but lacks information on return structure, pagination, or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25% (account described). Description explains format and provenance enum values with specific details, adding value beyond schema. However, conversion_id lacks description, and account minimum is not explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch' and the resource 'converted transactions for a completed conversion'. It distinguishes from sibling tools like get_conversion (likely metadata) and convert_bank_statement (initiation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after conversion completion but does not explicitly state when or compare to alternatives. No exclusion criteria or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversionsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It clarifies ordering and scope but does not mention read-only nature, pagination, or error cases. The description is adequate but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, front-loaded, and every sentence adds value. It efficiently communicates the tool's purpose and usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic purpose and usage but lacks details on the output format or pagination. Given the absence of an output schema, the description should provide more context on what is returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter (limit) with constraints but no description. The tool description does not mention this parameter or explain its purpose. The description fails to add value beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists the team's recent conversions, sorted newest first, and specifies its use for finding a conversion ID. It effectively distinguishes from sibling tools that perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the tool's purpose (finding a conversion id) which implies its usage context. However, it does not explicitly mention when not to use it or compare with sibling tools like get_conversion or get_conversion_result.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_bank_statementA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses cost per page, async behavior, and polling details. However, it does not cover potential errors, file size limits, or what happens if the file is invalid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, no unnecessary words, front-loaded with the core action. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the return value (conversion id) and the follow-up actions. It lacks details on error handling or file limitations, but covers the key workflow adequately for a low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters with 100% coverage. The description adds context about 'downloadable' and cost per page, but the cost is more of a behavioral note than parameter semantics. Minimal added value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it converts a PDF bank statement with CapyParse, specifying the input options (file_path or file_url). It distinguishes from sibling tools by outlining the asynchronous workflow with polling and result retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to pass a local file_path or downloadable file_url, mentions cost (1 credit per page), and provides a step-by-step process: returns conversion id, poll get_conversion until completed, then call get_conversion_result. The typical polling time and wait interval are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_creditsA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavioral traits. It adds the credit definition and usage hints but does not state that the tool is read-only or non-mutating. For a simple parameterless tool, the description is adequate but minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. The purpose is front-loaded, and the usage hint is appended succinctly. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is fairly complete. It tells the agent what the tool does and when to use it. Lacks details about the return format, but that is likely straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description adds value beyond the schema by explaining the credit unit, which relates to the tool's output rather than input parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Check remaining CapyParse credits' with a specific verb and resource. Explains the credit unit (1 credit = 1 statement page), distinguishing it from sibling conversion tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage scenarios: 'Call before converting large documents or after an insufficient-credits error.' This gives clear context for when to invoke the tool, effectively guiding the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.