Server Quality Checklist

Tool Scores

• parse_statementA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations already indicate the tool is read-only, idempotent, and non-destructive, the description adds behavioral context by stating that the limit parameter truncates returned rows but the full transaction_count is always reported, and that a ValueError is raised for unsupported formats. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with distinct sections for purpose, usage guidance, and parameter descriptions. It is concise (about 120 words) and front-loads the main purpose. The parameter descriptions are slightly redundant with the schema but useful for quick reference.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, output schema present, and 4 sibling tools), the description covers all essential aspects: input, behavior, return values ('A dict with the resolved format, columns...') and error conditions. It does not need to repeat output schema details since that is provided separately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining that filename extension selects format when format is omitted, that limit is optional and row count is always reported, and that content is inline text. It also mentions the default filename 'statement.xml'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool parses an inline statement payload into transaction rows and a summary, using a specific verb ('Parse') and resource ('statement transactions and summary'). It distinguishes itself from sibling tools ('summarize_statement' and 'validate_statement') by specifying when to use each alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool ('to read the full transaction detail plus the statement balances') and when to use alternatives ('When you only need the balances/totals use summarize_statement instead, and to merely confirm a payload parses use validate_statement'). This provides clear decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_formatA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the bar is lower. The description adds that it resolves format from content and filename, and raises ValueError if undetected, which is useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with clear sections (Args, Returns, Raises). Every sentence earns its place; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity and 100% schema coverage with output schema, the description fully covers what the tool does, its inputs, output, and error conditions. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. The description adds meaning by noting content is inline (not file path) and filename's extension is primary hint, along with defaults and recognized extensions. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it detects which bank statement format an inline payload is, using specific verb 'detect' and resource 'format'. It distinguishes from siblings by referencing parse_statement for subsequent actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use (when you hold statement text but do not yet know its format) and what to do next (call parse_statement instead of calling again). Provides clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_supported_formatsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds value by specifying the return value (format identifiers) and directing to a resource for human-readable details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise three-sentence description. Front-loaded with the main action, each sentence serves a purpose: states function, gives usage guidance, and describes return value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema exists, the description is complete. It explains what the tool returns, how it fits with sibling tools, and where to find additional info.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in schema, so description doesn't need to explain them. It adds context about what the tool returns, meeting the baseline for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List every bank statement format identifier this server can parse.' Uses specific verb and resource, and distinguishes from siblings by mentioning it's a precursor to detect_format and parse_statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Use this first to discover the valid format strings before calling detect_format or parse_statement.' Also references an alternative resource for more detailed information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_statementA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds that it 'never raises on a parse error' and returns a structured pass/fail with transaction count. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise with a clear structure: description, usage guidance, Args, Returns. Every sentence adds value. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown), the description explains return values (is_valid, format, transaction_count, error). All necessary context for a dry-run validation tool is provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for each parameter. The description's Args section adds context: 'content' is inline, 'filename' extension selects format, 'format' overrides detection. This adds meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Dry-run parse an inline statement to check it parses cleanly.' It uses a specific verb ('validate'/'dry-run') and resource ('statement'), and distinguishes from siblings like parse_statement ('to actually read the transactions') and detect_format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use: 'Use this to confirm a payload is well-formed and parseable before committing to a full read.' Also contrasts with the alternative parse_statement: 'To actually read the transactions use parse_statement.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_statementA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, non-destructive behavior. The description adds valuable context: it returns Decimal values stringified, raises ValueError for unsupported formats, and clarifies the inline nature of the input. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with clear sections (Args, Returns, Raises). Every sentence serves a purpose—no redundancy, and the key purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 params, 1 required, high schema coverage, and output schema mentioned), the description fully covers input requirements, output format, and error conditions. There are no missing gaps for an agent to use this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by clarifying that content is 'inline (not a file path)' and that filename extension selects format, which is not fully captured in the schema. This extra detail justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('Summarize an inline statement's balances and totals') and explicitly differentiates from the sibling tool parse_statement, which provides full transaction detail. This leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('when you need just the opening/closing balances... without the per-transaction rows') and provides a clear alternative ('For the full transaction detail... use parse_statement instead'). This gives agents actionable guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.