Server Quality Checklist

Tool Scores

• get_conversionA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds context beyond annotations by explaining output modes (inline JSON, local file save over stdio, HTTP download instructions). However, it does not disclose potential side effects or whether repeated polling affects the conversion state. The annotations (readOnlyHint: false, openWorldHint: true) signal uncertainty, but the description does not fully address behavioral traits like idempotency or changes to the conversion state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences with no redundancy. The first sentence states the purpose, the next two explain behavior in different modes, and the last gives a usage hint. Every sentence adds value, and it is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (polling tool with output schema and sibling tools), the description covers output modes and when to use it, but it lacks guidance on polling frequency, lifecycle (one-time or repeatable), and failure handling. The existence of an output schema reduces the burden for return values, but more context on the polling workflow would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and each parameter has a clear description in the schema (job_id, output_path, result_type). The tool description does not add any additional parameter semantics beyond what the schema already provides. With full coverage, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets the current state of one MainBook conversion, using a specific verb ('Get') and resource ('one MainBook conversion'). It effectively distinguishes from siblings: convert_bank_statement is the preceding step, list_conversions lists all conversions, and get_balance is unrelated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Use this after convert_bank_statement times out.' This tells the agent exactly when to invoke this tool. While it does not explicitly state when not to use it or list alternatives beyond the sibling set, the context is clear enough for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_bank_statementA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=false, idempotentHint=false, destructiveHint=false, openWorldHint=true. The description adds value by explicitly stating the workflow creates a job, spends page credits, and is not read-only. It does not contradict any annotation and provides useful behavioral context beyond the boolean hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences that are front-loaded and highly efficient. The first sentence immediately conveys the action and workflow; the second adds critical behavioral context. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (multi-step workflow, 6 parameters, output schema exists), the description covers the high-level workflow and side effects. It could briefly mention that results can be inline JSON or file-based (from result_type), but the parameter descriptions and output schema fill that gap. Overall complete for an agent to understand purpose and side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The overall description does not add parameter-specific meaning, but the individual parameter descriptions are already thorough. The tool description appropriately focuses on the overall workflow rather than repeating schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Convert one PDF bank statement' with a specific verb and resource, and outlines the complete workflow (create, upload, start, poll, return). It explicitly distinguishes itself from read-only siblings (get_balance, get_conversion) by stating 'it is not read-only' and 'spends page credits'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the primary conversion tool and notes it is not read-only, giving clear context for use. However, it does not explicitly state when not to use it or reference alternatives like list_conversions or get_conversion for post-processing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balanceA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, indicating a read-only, externally mutable resource. The description adds clarity by specifying the exact credits (total, reserved, available) and confirming the unit (PDF pages). No contradictions found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, dense sentences with no wasted words. The first sentence states what the tool returns, the second clarifies the measurement unit. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with an output schema, the description fully covers the purpose, items returned, and units. The output schema presumably details the structure, so no additional return-value explanation is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and there are no parameters to document. The description provides the meaning of the return values (total, reserved, available) which is helpful, but since there are no params, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and identifies the resource ('MainBook credits') and three precise items (total, reserved, available). It distinguishes itself from siblings like 'convert_bank_statement' or 'list_conversions' by being clearly a balance/account query tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when an agent needs to check credit balances before performing PDF-related operations. It does not explicitly state when not to use it or name alternatives, but with 0 params and a dedicated name, its niche is obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_conversionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true. The description adds behavioral context beyond that: it clarifies scope ('visible to the MainBook account') and the cursor-based pagination mechanism. This extra detail is valuable for an agent deciding how to interact with the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero wasted words. The first sentence states the core purpose and scope; the second gives the key usage instruction for pagination. Information is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (pagination list with two parameters), full schema coverage, presence of an output schema, and comprehensive annotations, the description is complete. It does not need to explain return values (output schema covers that) and provides all necessary usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both limit and cursor. The description mentions 'cursor page' and 'next_cursor,' reinforcing the cursor parameter's role but adding no new semantic detail beyond what the schema already provides. Baseline score 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List one cursor page of conversion jobs visible to the MainBook account.' It uses a specific verb ('list'), resource ('conversion jobs'), and includes scope constraints ('one cursor page', 'MainBook account'), effectively distinguishing it from sibling tools like get_conversion (single item) and convert_bank_statement (action).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description instructs the agent to 'Pass the returned next_cursor to continue,' providing clear pagination usage. It implies the tool is for listing pages of conversions but does not explicitly state when not to use it or compare to alternatives. However, given distinct siblings, the guidance is effective.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• output_folderA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotent and non-destructive. Description adds context about inspecting vs changing, 'allowed folders' restriction, and special 'next_to_source' value. This enriches the behavioral model beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no filler. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter, output schema present, and clear annotations, the description covers all needed context: read vs write, allowed folder restriction, reset behavior. No missing information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description aligns perfectly. Both clarify that omitting path reads, providing a path changes it, and 'next_to_source' is a special reset value. No gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read or change the default local result folder' with specific verbs and resource. It distinguishes from siblings which deal with bank statements and balances, so no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Call with no path to inspect the current setting' for read, 'Pass an allowed absolute folder, or 'next_to_source' to restore' for write. No sibling overlap requires exclusion clauses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.