Server Quality Checklist

Tool Scores

• get_required_fieldsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the tool is clearly safe and non-destructive. The description adds no additional behavioral context beyond stating it lists fields, which is appropriate. With annotations covering the safety profile, a score of 3 is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at 4 lines, front-loading the purpose and then providing usage guidance. No unnecessary information. Slightly more verbose than necessary but still well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, output schema exists, low complexity), the description adequately covers purpose, usage, and parameter. No need to detail return values since output schema is available. Complete for its context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description's parameter documentation ('message_type: A supported ISO 20022 acmt message type') adds no extra meaning beyond what is already in the schema's description and enum list. Baseline 3 is correct.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List only the required input field names for an acmt message type' with a specific verb (list) and resource (required field names). It explicitly distinguishes from the sibling tool get_input_schema by noting that the sibling provides full type/format constraints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('quick checklist of mandatory columns before building account records') and when not to use it ('when you need full type/format constraints, call get_input_schema instead'), providing clear guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_messageA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint, idempotentHint, destructiveHint. The description adds that no file is written, records must be in memory, and returns XML string or error payload. This supplements annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: first sentence states purpose, then adds context, usage tips, return value, and args. Every sentence provides essential information with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (not shown), but the description explains return values clearly. It covers prerequisites and no file writing. For a simple 2-param tool with good annotations and schema, the description is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description's 'Args' section mostly repeats schema info. It adds marginal value by emphasizing 'flat' records and the prerequisite to validate, but these are already in schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Generate a validated ISO 20022 acmt XML message from in-memory records' with specific verb and resource. It distinguishes itself from siblings by noting it is the primary generation tool and referencing prerequisites like validate_records and list_message_types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises running validate_records first and list_message_types to confirm message_type, providing clear usage context. It implicitly suggests when not to use this tool (e.g., before validation), but could be more explicit about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_input_schemaA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, establishing the tool as a safe, idempotent read operation. The description does not add additional behavioral context beyond what is already captured by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. Each sentence adds value, and the structure is clear with a brief usage note and sibling references.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with rich annotations and schema, the description covers purpose, usage, and alternatives. It does not explicitly state that the return is a JSON Schema object, but this is implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the parameter message_type already has a detailed description and enum in the schema. The description repeats the parameter info without adding new meaning, so it meets the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full JSON Schema for a message type's flat input record. It explicitly distinguishes from sibling tools by mentioning alternatives for required fields (get_required_fields) and validation (validate_records).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this to learn every field, its type, and its constraints before assembling records, or to drive a form/UI.' It also specifies when not to use it by directing to siblings for specific cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_identifierA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the description's role is lighter. It adds the return format {'kind', 'value', 'valid'}, which is useful context for the agent. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise (5 sentences) with front-loaded purpose, no wasted words. Every sentence adds value, including the return format and parameter list.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple validation tool with 2 parameters and no output schema, the description fully covers what the tool does, when to use it, and what it returns. The explicit return object compensates for the lack of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description restates the parameters with minor additions (case-insensitivity already in schema), but does not add significant new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Validate' and the resource 'single financial identifier (IBAN, BIC, or LEI)', distinguishing it from the sibling tool 'validate_records' which handles batch validation. It is specific and leaves no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this for a one-off identifier check' and directs to use 'validate_records' for batch records, providing clear when-to-use and when-not-to-use guidance with a named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_message_typesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly and idempotent, so the description's value is in detailing the return format (list of dictionaries). This adds context beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-loading purpose, usage, and output. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and an output schema, the description sufficiently describes the return format and purpose. No gaps for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; baseline 4 applies. The description correctly does not add parameter info as none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists every supported ISO 20022 acmt message type with human names, and distinguishes itself from sibling tools like get_required_fields and get_input_schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to use this tool first before generation/validation calls, and provides alternatives for fetching field schemas (get_required_fields, get_input_schema).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_recordsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, indicating safe, non-destructive operation. The description complements this by specifying the exact return format ({'valid', 'total', 'valid_count', 'errors'}), which adds valuable behavioral context beyond annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two short paragraphs and a structured 'Args' section. Every sentence adds value: purpose, usage guidance, return type, and parameter summaries. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description explicitly provides the return format, making it complete. It also references sibling tools (get_input_schema, get_required_fields) for further context. Given the tool's simplicity (2 required parameters), this is fully sufficient for an AI agent to understand and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters in detail. The description provides a brief summary of each parameter (e.g., 'A supported ISO 20022 acmt message type') but adds little beyond what the schema includes. It mentions links to related tools, which is helpful, but not enough to elevate above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Validate') and resource ('flat account records against a message type's input JSON Schema'). It clearly distinguishes the tool from siblings like validate_identifier and generate_message by stating what it checks (JSON-Schema shape) and its intended use before generate_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('before generate_message') and when not to ('to validate a single financial identifier... use validate_identifier'). It also clarifies that it only checks JSON-Schema shape, setting proper expectations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.