Server Quality Checklist

Tool Scores

• get_config_with_inline_diffA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It adds behavioral detail like 'inline change markers' and 'node may be an exact name or a unique partial name,' which is useful. However, it does not disclose read-only implications, error behavior, or what happens with invalid reference versions, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the primary functionality and free of unnecessary detail. Every phrase contributes to understanding the tool's behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema and 100% parameter schema coverage, the description provides enough context for selecting and invoking the tool. It covers the main behavior and node matching but lacks explicit comparison to sibling tools, which prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description only partially restates the node parameter's matching behavior already documented in the schema, adding no new semantic value beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the complete current configuration with inline change markers relative to a reference version, using a specific verb and resource. It conveys the core function but does not explicitly differentiate itself from siblings like get_diff_between_versions, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: when you need the current config with inline diff against a reference version. However, there is no explicit mention of when to use this tool over alternatives such as get_diff_between_versions or prepare_config, and no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diff_between_versionsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does disclose the output format (unified diff) and mentions that node matching allows partial names, but it does not address error handling, version ordering, or ambiguous matches. This adds some context but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and immediately states the primary purpose in the first sentence. The second sentence adds one key constraint about node matching. Every word earns its place, with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema and output schema cover parameters and return format, so the description is adequate for a basic call. However, it lacks guidance on edge cases like version ordering, ambiguous partial-name matches, or how the group parameter affects the lookup. Given the tool's moderate complexity and lack of annotations, the description is only moderately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already well described. The tool description only repeats the node-matching detail already present in the schema and adds no new parameter semantics. Thus, it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a unified diff between two historical versions of a device configuration, which is a specific action with a clear resource. This distinguishes it from sibling tools like get_device_versions (which lists versions) and get_config_with_inline_diff (which likely compares against current config).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not explicitly state when to use it versus alternatives, nor does it provide exclusions or prerequisites. The usage context is implied by the action, but no direct comparison to sibling tools is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_statusA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses the partial-name matching behavior but does not detail error handling for ambiguous or unmatched names, nor clarify what 'status details' include. This is a useful but limited behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, front-loaded with the core purpose and adding a relevant usage nuance. Every word earns its place with no unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-device status lookup, the description covers purpose and the key input behavior. The output schema presumably documents the return value. It lacks explicit failure behavior for ambiguous partial names, making it just short of complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes both parameters (node and group), and the tool description only restates the node matching semantics present in the schema. No additional parameter-level meaning is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return status details for a single device,' using a specific verb and resource. The note about partial name matching distinguishes this from sibling tools that list or find devices, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when status details for a single device are needed, and the partial-name note provides a usage hint. However, it does not explicitly contrast with alternatives like list_devices or find_devices, nor specify when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_blameA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that the tool returns only device metadata and points to another resource for the annotated config, but it does not clarify whether 'prepare' has side effects, requires specific permissions, or is idempotent. This is partial but not full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the tool's purpose, and every sentence contributes: what it does, node flexibility, and where to get the final result. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, full schema coverage, and an output schema, the description is adequately complete. It tells the user what to expect (metadata only) and directs to the follow-up resource, though it could briefly explain why this preparation step is needed versus calling get_blame directly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to restate parameters. It adds the nuance that 'node' can be a unique partial name, but this is already present in the schema description. No additional parameter-level meaning is provided beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair: 'Prepare one device's configuration blame' and identifies the target resource ('static get_blame'). It clearly distinguishes from siblings by focusing on blame preparation rather than configuration retrieval or diffing, and notes it returns only metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: as a preparation step for get_blame, with node as exact or unique partial name. It directs the user to read oxidized://config/get_blame for the annotated config, effectively indicating the follow-up resource, though it does not name alternative sibling tools explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_configA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns only device metadata and not the configuration, and that node can be an exact or unique partial name. Yet it does not explain side effects (if any), failure modes, or permission needs, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose, node matching rule, and return type/next step. Front-loaded and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description need not detail return fields. It clearly explains the tool's role relative to get_content, but omits error behavior and when group might be needed. Still, it is sufficient for a simple preparatory tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description repeats the node matching behavior already in the schema ('exact name or a unique partial name') but does not add meaning beyond it, such as how group interacts with partial names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: preparing one device's current configuration for the static get_content resource. It uses a specific verb ('prepare'), identifies the resource ('device's current configuration'), and distinguishes itself from siblings by tying to get_content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it is a preparatory step for get_content and directs users to read the actual configuration from oxidized://config/get_content. However, it does not explicitly mention alternatives or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It discloses that the tool returns all devices and enumerates the output fields, but it does not mention potential pagination, sorting, response size, or explicitly confirm read-only behavior. The absence of such details when there are no annotations leaves a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: the first states the action, the second lists the returned fields. It is front-loaded, free of redundant information, and every sentence contributes meaning, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter list tool, the description adequately specifies the return structure and scope. It could be more complete by referencing sibling tools for filtered queries or noting any pagination/performance characteristics, but these are not essential for correct invocation. The presence of an output schema (even though not explicitly shown) and explicit field listing makes it fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100%, so there are no parameter specifics to elaborate. The description instead adds value by explaining the output structure (fields), which is the relevant semantic content for a no-input list tool. Baseline of 4 for zero-param tools is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'all devices managed by Oxidized', which explicitly defines the tool's scope. It lists the included fields (name, model, group, ip, last backup time, status), making it unambiguous and distinguishing it from sibling tools that target specific devices or configurations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool is for retrieving the complete device list, which implies its use case. However, it does not explicitly contrast with alternatives like find_devices for filtered queries or get_device_status for single-device status, so it lacks explicit when-not/exclusions guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_versionsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description takes on full responsibility for behavioral disclosure. It adds meaningful details: versions are ordered oldest-first, version numbering semantics, and that each entry includes timestamp and OID. This goes beyond a simple 'returns versions' statement, though it does not cover permission requirements or error cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary purpose, then provides essential details in a structured order. Every sentence carries useful information without redundancy. It is highly concise and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's modest complexity, the description covers the key aspects: what is returned, how the node is interpreted, ordering, and content of each entry. An output schema exists, so detailed return-value documentation is not strictly needed. Missing details like the exact output schema or edge-case behavior are minor, and the description is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents both parameters (node, group) with high coverage. The description adds value by explaining that node can be an exact name or a unique partial name, which clarifies the matching behavior beyond the schema's 'full name or unique part'. The group parameter is not mentioned, but the schema covers it, so the description supplements rather than repeats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'all available configuration versions for a device'. It also clarifies partial-name matching, which distinguishes it from sibling tools like list_devices or get_device_status that deal with device lists or status rather than version history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case: retrieve configuration versions for a device by name. It provides context about node matching (exact or unique partial) but does not explicitly mention when to choose this over sibling tools like get_diff_between_versions. There is no exclusions or alternative guidance, but the context is sufficient for a straightforward listing tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_devicesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It discloses relevant behavior: 'Matching is case-insensitive' and search by exact/partial name or IP. While it doesn't mention no-match behavior or pagination, the output schema exists and this is a low-risk read-only search, making the disclosure adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and includes the key behavioral detail (case-insensitive) plus usage context. Every sentence earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with two parameters, an output schema, and clear usage guidance, the description is complete. It explains what the tool does, when to use it, and a key matching behavior. No missing critical information that would hinder correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds minimal semantic value beyond the schema: it clarifies 'exact or partial' matching, but the schema already lists query as 'part of a device name, full device name, full group/name, or IP.' The 'group' parameter is only in the schema, not the description, so no new meaning is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find Oxidized devices by exact or partial name, full name, or IP address.' It uses a specific verb ('find'), identifies the resource ('devices'), and specifies search criteria, distinguishing it from sibling tools like 'list_devices' which likely lists all devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use this before other device tools when only part of a device name is known.' This clearly signals the tool's role in the workflow and differentiates it from alternatives like listing all devices.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.