Server Quality Checklist

Tool Scores

• list_devicesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false, and the description's 'List' verb is consistent with these. The description adds the ability to choose query_type and the details flag, plus a return type statement, but does not disclose additional behavioral traits such as pagination, error handling, or filter semantics. It is not contradictory, but could be richer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a short intro, a consolidated note, and an Args/Returns section. However, the Args list repeats schema content, making it more verbose than necessary. It is not overly long, but not as concise as it could be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the read-only nature, six optional parameters, and the existence of an output schema, the description covers the core functionality, query modes, and filter options well. It explains the return type and the details flag, but lacks explicit guidance on when to prefer this over sibling tools or how filters interact. Overall, it is sufficiently complete for a listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already well-documented in the input schema. The description's Args list largely duplicates this information, adding no new meaning beyond examples like 'cisco_ios' and 'arista_eos' that already appear in the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List network devices and inventory information' with a specific verb and resource, and further explains the consolidated access to devices, groups, or all. While it does not explicitly contrast with sibling tools, the purpose is unambiguous and matches the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives some usage context ('Consolidated tool', 'Use details=true for full device attributes') but does not explicitly state when to use this tool versus alternatives like show_commands or fetch_data. No exclusions or alternative tool references are provided, so guidance is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_configA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true and readOnlyHint=false, so the description adds value by disclosing the return format (dict mapping hostname to success/error). It does not contradict annotations, and it provides a concrete behavioral detail beyond the schema and annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with a clear Args/Returns structure. The Args list is somewhat redundant with the schema, but the overall format is efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and the input schema covers all 5 parameters with descriptions, the description is complete enough for basic usage. It covers the main command and filter options, though it does not explain how multiple filters combine or mention potential non-deterministic behavior (openWorldHint).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description's Args section mostly repeats the schema's parameter descriptions. No additional semantic meaning is added beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Send configuration commands to network devices' with a specific verb and resource, clearly distinguishing it from read-only siblings like show_commands and backup_configs. The Args and Returns sections further clarify that this applies configuration changes and returns per-host results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for applying configuration commands versus alternatives like show_commands, but it does not explicitly state when to use this tool or when to prefer a sibling. No exclusions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• backup_configsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotent, non-destructive behavior (destructiveHint=false). The description adds that it writes to local disk and filters by inventory attributes, which is valuable context. Yet it does not clarify overwrite behavior, file naming, or whether authentication/network access is needed, so the disclosure is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with a front-loaded one-line purpose followed by a compact parameter list and return note. Every sentence earns its place, and there is no fluff or repetition of schema details beyond what is necessary for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, annotations, and full parameter descriptions, the tool is well-specified for a straightforward backup operation. However, it does not explain how multiple filters combine (e.g., AND vs OR) or specify whether filters are case-sensitive, which could affect invocation. Still, the essential context is present, so it is slightly above minimum viability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100%, with each parameter already described clearly (e.g., filter_name as 'Filter by device name'). The description merely restates the parameter names and one example for filter_platform without adding new syntax, precedence, or combination semantics. Baseline 3 applies because the schema fully documents the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Save') with a clear resource ('device configuration') and destination ('local disk'). It distinguishes itself from sibling tools like apply_config, which pushes configurations rather than saving them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the tool is for backing up device configurations to local disk, which signals appropriate use cases. However, it does not explicitly mention when not to use it or contrast it with alternatives like fetch_data or show_commands, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• show_commandsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, lowering the burden. The description adds that execution happens via SSH and describes the return structure as a dictionary mapping hosts to success/error results, providing useful behavior beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with an Args section and a Returns section. Every sentence adds relevant information without redundancy, and the main purpose is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (multiple filters, raw command execution), the description covers all necessary aspects: purpose, parameters, execution method, and return format. The output schema already provides structure, and the description fills in the behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the parameters, so the description need not repeat them. It adds no extra semantic detail beyond what the schema already provides, but the schema descriptions are sufficient. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes raw CLI show commands via SSH, combining a specific verb ('execute') with a concrete resource ('raw CLI show commands'). It distinguishes itself from siblings like apply_config (config changes) and backup_configs (backups) by emphasizing the read-only 'show' nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly establishes the intended context: raw CLI show commands over SSH. It does not explicitly mention when not to use it or name alternatives, but the 'show' scope and SSH method provide clear contextual guidance for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_dataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety profile is covered. The description adds useful behavioral context by enumerating available getters and stating the return format ('Structured data per host mapping hostname -> result'). It does not disclose caveats like credential requirements or performance implications, but given the annotations, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a one-sentence purpose, a bulleted list of common getters, a labeled Args section, and a Returns section. Everything earns its place, and the most important information is front-loaded. It is detailed yet not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, output schema present, read-only annotations), the description is largely complete: it explains what the getters return, lists parameters, and describes the output mapping. The only gap is the lack of detail on getters_options, but the presence of an output schema and annotations reduces the burden.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 83%, the baseline is 3. The description enhances parameter understanding by giving concrete examples of valid getter names (e.g., 'facts', 'interfaces') and noting that getters_options is optional, although it remains vague about its structure. This adds value beyond the schema's generic parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb-resource pair: 'Execute one or more NAPALM getters to retrieve structured data from network devices.' This strongly distinguishes it from siblings like apply_config (write) and list_devices (inventory listing). The list of common getters further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states the tool's function and provides a list of common getters, which implies when to use it (e.g., when you need facts, interfaces, BGP state). However, it does not explicitly name alternatives or exclusions (e.g., 'use show_commands for raw CLI output'), so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.