Server Quality Checklist

Tool Scores

• get_iban_formatA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, open-world. Description adds that unsupported country codes return an error describing the problem. No contradictions; adds value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise two-paragraph structure; first sentence front-loads purpose. Every sentence provides distinct value: scope, return contents, usage guidance, error behavior. No redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with one parameter and no output schema, the description adequately describes return fields (IBAN length, BBAN layout, example, SEPA flag) and error behavior. Could be more precise about error format, but sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameter description with length, pattern, examples. Description does not add additional semantic info about the parameter beyond what's in the schema. Baseline 3 appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'return' and resource 'IBAN format specification' with explicit mention of 90 countries. Distinguishes from siblings by specifying it's for format specs, not validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states use cases (understanding structure, building input masks, explaining validation failures) and explicitly contrasts with validate_iban for validation. Provides clear when-to-use and when-not-to.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_bicA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare safe/read-only behavior. Description adds specific return fields and clarifies that unknown or malformed BICs return an error, not a fabricated guess. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose, followed by concise details and usage. Every sentence adds value; no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without output schema, description enumerates all return fields and explains error behavior. Tool is simple (one param) and description covers everything needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers parameter fully (100% coverage). Description reinforces length rules, gives concrete examples (DEUTDEFF, DEUTDEFF500), and distinguishes head office vs branch BICs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb (look up) and resource (financial institution by BIC), specifies return details (bank name, city, etc.), and immediately distinguishes from sibling IBAN tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use: to resolve BIC, confirm SWIFT code, enrich IBAN. Does not exclude alternatives, but context is clear. Sibling tools are unrelated (IBAN-focused), so no confusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ibanA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. Description adds that no data is stored and validation is in-memory, and clarifies error handling: returns valid:false with reason instead of throwing. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient: first sentence states purpose, second explains checks, third describes output, fourth provides usage guidance. No useless sentences, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, high schema coverage, and no output schema, the description fully covers return fields, error behavior, and data handling. Complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and describes case-insensitivity and space tolerance well. Description does not add extra meaning beyond confirming the parameter is an IBAN. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb+resource: 'Validate a single IBAN against official ISO 13616 structure.' It distinguishes from sibling tools like validate_bulk_ibans and extract_ibans_from_text, which are mentioned explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use ('Use this when you have one account number to verify'), when-not-to-use ('For many IBANs prefer validate_bulk_ibans'), and alternatives ('to pull IBANs out of prose use extract_ibans_from_text').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_ibans_from_textA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the description doesn't need to repeat safety. It adds valuable details: input is processed in memory and not stored, and text with no IBAN returns empty list rather than an error. This goes beyond annotations, but could mention validation criteria or output format more explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence serves a purpose: main action, use cases, output behavior, relationship to sibling, privacy. No redundant phrases, front-loaded with key verb and resource, and logically organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description is remarkably complete. It covers input types, edge cases (no IBAN), output structure (JSON array with validation result), privacy properties, and relationship to sibling tools. The agent has all info needed to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with description for 'text' parameter. The description adds semantic value beyond the schema by explaining that text can be arbitrary, IBANs may be split across spaces, and it gives examples of input types (emails, invoices). This helps the agent understand acceptable input formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans free-form text, extracts candidate IBANs, validates them, and returns results. The verb 'extract' paired with resource 'IBANs from text' is specific, and it distinguishes itself from sibling tools like 'validate_bulk_ibans' by being the first step for messy text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: unstructured sources like emails, invoices, PDFs, chat messages. It also says when not to use (no IBAN returns empty list) and suggests using 'validate_bulk_ibans' as a follow-up if needed. Provides clear alternatives and context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_bulk_ibansA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnly, not destructive, idempotent, and open world. The description adds behavioral details beyond annotations: validation happens in memory, no storage, preserves input order, returns a summary count, and produces per-IBAN results with optional reasons and bank details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise and well-structured: first sentence states core action and limit, second details return format, third provides usage scenario and additional notes. No redundant sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully explains the return structure (array with valid, countryCode, reason, bank details, summary count). Combined with thorough annotations and parameter semantics, the description is complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'ibans' has 100% schema description coverage. The description adds semantic details: case-insensitivity, tolerance of spaces, and order preservation in the response, which are not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as validating a batch of up to 100 IBANs using ISO 13616 checks, distinguishing it from the single-IBAN sibling `validate_iban` by explicitly mentioning batch processing and listing the same checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Use this instead of calling validate_iban in a loop' when checking a list, and advises splitting inputs larger than 100. Also notes that account numbers are never stored, indicating privacy considerations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.