Server Quality Checklist

Tool Scores

• send_config_commandsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as destructive, and the description adds the return format (dictionary with 'hosts' key) and filter-based targeting behavior. However, it does not disclose potential side effects, ordering guarantees, or rollback behavior beyond the annotation's destructive hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with a clear purpose sentence, then structured lists for arguments and return value. Every part adds useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, all parameters, and return format, making it reasonably complete for a command-sending tool. It lacks usage guidance and additional caveats, but given the output schema and annotations, it provides sufficient context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes an 'Args' section that explains all five parameters, including the semantics of each filter (name, hostname, group, platform) and the commands list. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Send configuration commands to network devices' with a specific verb and resource, distinguishing it from sibling read-only tools like run_show_commands or list_network_devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided about when to use this tool versus alternatives such as run_show_commands for read-only operations or backup_device_configs for backups. The description simply defines the action without contextual usage notes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_show_commandsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the read-only nature is covered. The description adds that execution happens via SSH and that the return dictionary maps hostname to success or error. However, it does not disclose potential rate limits, command timeout behavior, or how raw output is handled, leaving some behavioral details uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the primary purpose, and structured with Args/Returns sections. Every sentence provides useful information without redundancy, and the formatting makes parameters and return values easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with five parameters and an output schema, the description adequately covers the return structure and filter parameters. It misses explicit note on how multiple filters interact and what 'raw' implies about output formatting, but these are minor given the tool's straightforward nature and the presence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries the burden. It provides meaningful one-line explanations for all parameters: commands, filter_name, filter_hostname, filter_group, and filter_platform. It adds practical context like 'e.g., cisco_ios' for platform filtering, though it does not clarify filter combination logic (AND/OR).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Execute raw CLI show commands via SSH.' This distinguishes it from siblings like send_config_commands (which writes configs) and run_napalm_getter (which uses a different retrieval mechanism). The 'show commands' qualifier makes the tool's scope explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for read-only show commands, but it does not explicitly state when to prefer this tool over alternatives like run_napalm_getter or list_network_devices, nor does it mention any exclusions. The 'show commands' phrasing provides contextual guidance but lacks direct comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• backup_device_configsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotentHint=true, readOnlyHint=false, and destructiveHint=false. The description adds that it returns a summary of saved file paths, but does not disclose behaviors like overwrite policy, directory creation, or naming conventions. It does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a well-structured docstring with clear sections for Args and Returns. Every sentence provides necessary information without fluff. It is concise and effectively front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and annotations, the description adequately covers purpose, parameters, and return value. It could mention prerequisites like needing an active connection to devices, but since all parameters are optional and the tool operates on an inventory, the description is sufficiently complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the schema itself has no descriptions (0% coverage), the description compensates by providing a concise Args block that explains each parameter's purpose and includes an example for filter_platform. This adds meaningful semantics beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Save device configuration to the local disk') and is clearly distinct from sibling tools like list_network_devices, send_config_commands, and run_show_commands. It summarizes the operation in one unambiguous sentence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states exactly when to use the tool: to save device configurations to disk. It also lists filters that imply use with an inventory. However, it does not explicitly contrast with alternatives or state when not to use it, though the action itself is clear enough to avoid confusion with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_napalm_getterA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already ensures safety, and the description adds useful behavioral context by enumerating common getters and stating the return format (structured per-host data). It does not contradict annotations and provides enough detail for a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for getter examples, arguments, and returns. Every sentence earns its place, and the bullet list of common getters is highly useful without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with annotations and an output schema, the description covers the key aspects: what getters can be used, how to filter hosts, and what the return value looks like. It is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining each parameter (getters, getters_options, and all four filters). It distinguishes filter_name vs filter_hostname vs filter_group vs filter_platform, though getters_options remains somewhat vague.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes NAPALM getters on network devices, which is a specific verb+resource combination. It distinguishes itself from siblings by focusing on NAPALM getters (structured data) rather than raw commands or config backup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for retrieving structured device data (facts, interfaces, etc.) but does not explicitly compare with alternatives like run_show_commands or backup_device_configs. There is no 'use this when' or 'instead of' guidance, leaving the choice to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_network_devicesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description is consistent with that. It adds useful behavioral details: the tool is 'consolidated', query_type controls the shape of data, and details applies only to devices query. This goes beyond the annotation by clarifying internal behavior and constraints, though it does not address potential side effects (none expected) or edge cases like filter combinations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a summary, a note on flexibility, an Args block listing all parameters, and a Returns line. It is appropriately sized for a tool with six parameters, and every line serves a purpose. It could be slightly tightened by removing the Returns line since an output schema exists, but it remains clear and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with six optional parameters and flexible output, the description covers the main functionality, parameter semantics, and the special case of details=true. It does not specify how filters combine (e.g., AND vs OR) or what 'full inventory attributes' includes, but the output schema likely covers return values. Overall, the description is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions (0% coverage), making the description the sole source of parameter meaning. The approach documents each parameter with concrete examples and defaults, e.g., query_type: 'Type of inventory data to return ("devices", "groups", "all")' and filter_platform: 'Filter by platform (e.g., cisco_ios)'. This fully compensates for the schema's lack of descriptions and adds practical value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List network devices and inventory information.' It further explains it provides flexible access to devices, groups, or both, which aligns with its name. The purpose is distinct from sibling tools like send_config_commands or run_show_commands, making it easily differentiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on what the tool is used for (inventory retrieval) and provides practical guidance like 'Use details=true for full device attributes.' It does not explicitly mention when to avoid this tool or cite alternatives, but the one-liner on details and the consolidated nature imply usage scenarios. Sibling tool names reinforce distinction, though no explicit 'use X instead' is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.