Server Quality Checklist

Tool Scores

• z2m_health_reportA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and idempotentHint annotations, the description adds valuable behavioral context: it generates no Zigbee traffic, returns a structured object with specific keys, and explains that an empty issues object means nothing is wrong. It also notes that collect_seconds causes the call to wait, which is a non-obvious behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense and well-organized, but slightly longer than necessary. The note about collect_seconds duplicates the schema, and the list of issue types, while helpful, could be more compact. Still, every section serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and absence of an output schema, the description provides a solid high-level overview: return keys, empty issues semantics, and parameter behavior. However, it does not explain the structure of sub-objects like bridge, totals, capabilities, or hints, leaving some ambiguity for an agent interpreting the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions collect_seconds but merely restates the schema's explanation about waiting; it adds no new semantic information. Thus it does not elevate the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: audit the entire Zigbee estate and return all attention-worthy issues, enumerating specific categories. It also differentiates from siblings by explicitly recommending this tool over z2m_check_updates and z2m_network_map for routine checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases (routine checks, answering 'is my Zigbee network healthy?') and names alternatives, but it does not fully clarify when those alternatives should be preferred. This is strong guidance but lacks explicit when-not conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_manage_groupA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate mutability (readOnlyHint=false) and non-destructive (destructiveHint=false). The description adds valuable behavioral context: removing a group does not affect devices, removal fails if a member is unreachable unless force is set, and with force the device retains membership internally. This goes beyond annotation basics, though it doesn't cover every action's side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary actions. Each sentence adds value: the group benefit, the removal caveat, and the workflow. No wasted words, and the structure guides the agent from understanding to action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has multiple actions and no output schema, but the description provides the core purpose, the critical force-removal edge case, and workflow tips. It omits details like 'new_name is required for rename' and 'device is needed for member actions', but the schema covers those. Given the complexity, the description is sufficiently complete for safe operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (83%), with descriptions for most parameters. The description adds context for 'force' (removal behavior) and implies the need for 'new_name' (rename) and 'device' (member actions). However, it doesn't systematically explain each parameter, so it meets the baseline but doesn't greatly exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Create or remove groups, rename them, and add or remove device members.' It uses specific verbs and identifies the resource (Zigbee groups), distinguishing it from sibling tools like z2m_list_groups and z2m_set_state by explicitly referencing them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides direct guidance on when to use this tool versus alternatives: 'Use z2m_list_groups to inspect groups first, and z2m_set_state to control a group once created.' It also explains the benefit of groups (multicast responsiveness), which helps the agent decide when group management is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_bridge_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already state readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description reinforces this with 'Read-only' and adds useful context about the return format (JSON object) and the large size of the full configuration when include_config is true. This goes slightly beyond the annotations, though it doesn't disclose any safety-relevant edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the purpose, then usage guidance, then parameter caution. Every sentence carries necessary information with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only status tool with one optional parameter and no output schema, the description is complete: it lists return fields, provides usage context, differentiates from siblings, and warns about the large config. No important information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter, so the schema already fully explains include_config. The description adds the hint to use it 'only if you need' the full config, which is mildly helpful but does not add substantive semantic meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('Zigbee2MQTT bridge status'), enumerating the exact attributes returned (version, coordinator, channel, PAN ID, etc.). It clearly distinguishes itself from siblings by positioning itself as the starting point and explicitly naming alternatives for different tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Start here to understand the estate' and tells the agent to use z2m_health_report for problems or z2m_list_devices for individual devices. It also advises to 'set include_config only if you need the full configuration', giving clear conditional usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_get_deviceA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the 'Read-only.' addition is somewhat redundant. However, the description adds value by listing the internal detail categories returned, which is behavioral context beyond the annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, all densely informative with no fluff. Each sentence serves a purpose: what it does, safety, prerequisites, and when to use an alternative. 'Read-only' is brief but not padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without an output schema, the description lists the key return categories and clarifies the tool's role as a discovery step before write operations. This is sufficient for an agent to decide when to call it and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters, and the schema clearly describes 'device' (friendly_name or ieee_address) and 'include_raw'. The description adds no additional parameter semantics beyond what the schema provides, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Get full detail for one device') and enumerates exactly what is returned (state, properties, options, endpoints, bindings, reportings). It distinguishes from siblings by explicitly contrasting with z2m_list_devices and stating this is for a single device with exact name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use and when-not-to-use guidance: 'Call this before z2m_set_device_options, z2m_bind or z2m_configure_reporting' and 'Use z2m_list_devices instead when you want many devices or do not know the exact name.' No ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_bindA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, destructiveHint=false), the description discloses critical behavioral traits: the effect of each action (unbind reverses, clear removes all binds at once), the endpoint syntax, and the requirement that both devices must be awake. This gives the agent a solid understanding of side effects and preconditions, exceeding what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is five sentences long, but every sentence carries useful information: purpose, benefit, action semantics, endpoint guidance, and a precondition. It is front-loaded with the core purpose and then adds relevant details. Slightly dense, yet no sentence is wasted, so it earns a 4 rather than 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no output schema, this description is exceptionally complete. It covers all actions, endpoint handling, prerequisites, and the practical impact of binding (works without coordinator). The agent has enough information to invoke the tool correctly and anticipate results, even without output schema details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 75% schema coverage, the description must enrich parameter meaning. It does so by explaining the /ENDPOINT syntax for 'from' with an example, noting that 'to' is unnecessary for clear, and clarifying that action='clear' has a bulk effect. This adds value over the bare schema, though it doesn't elaborate on cluster name formats beyond the schema's example.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pairing: 'Bind or unbind clusters between two devices, or between a device and a group.' This clearly distinguishes the tool from siblings like z2m_manage_group or z2m_configure_reporting, which handle different operations. It also explains the underlying purpose of binding, making the tool's role unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong usage context: it explains the benefit of binding (works even if the bridge is down), gives a prerequisite ('check z2m_get_device endpoints first'), and notes device wakefulness. However, it does not explicitly name alternative tools or state when not to use binding, so it falls short of a 5. The context is clear enough for an agent to decide when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_list_devicesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds the behavioral nuance that collect_seconds will wait before answering, and notes the tool is read-only despite the waiting. This goes beyond the annotations by explaining the side effect of a parameter, but it doesn't delve into other behaviors like pagination or failure modes. Given annotations cover safety, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, starting with the core action and resource. Each sentence serves a purpose: the field list, the read-only note, the use cases, the alternative guidance, and the return format. No filler words; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description includes the return structure '{total, returned, capabilities, devices[]}', which is essential given no output schema exists. It also covers common user intents (offline devices, weak signal) and distinguishes from sibling tools. While it doesn't explain every parameter, the schema handles that. This is reasonably complete for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (88%) with descriptions for most parameters. The description adds value by explaining that collect_seconds waits to populate live fields like linkquality, battery, and OTA, and hints at the 'weakest signal' use case which aligns with sort_by and only_problems. It doesn't repeat each schema description but enriches the meaning of key parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description immediately states the tool lists Zigbee devices with administrative detail and enumerates the specific fields (type, model, vendor, power source, link quality, battery, availability, last_seen, interview state, OTA updates). It also distinguishes itself from siblings by naming z2m_health_report and z2m_get_device as alternatives, making its unique purpose clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases: 'Use this to answer which devices are offline or which have the weakest signal'. It also provides clear alternatives with specific conditions: 'Prefer z2m_health_report for a prioritised summary of everything wrong at once, and z2m_get_device when you need one device's exposes, options or bindings.' This is excellent when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_check_updatesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses real network traffic, per-device time cost, and default behavior (checks all mains-powered routers when no arguments). This goes beyond the annotations by adding practical side-effect context, without contradicting readOnlyHint/idempotentHint/destructiveHint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each with clear purpose: main action, side effect caveat, default behavior, and usage/alternative guidance. No wasted words, front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully covers when to use, what happens, and what to do next. No output schema is needed for this simple action, and annotations cover safety. The combination of description, schema, and annotations gives the agent everything required for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with parameter descriptions already explaining the device field and timeout. The description's note about omitting arguments checks all mains-powered devices adds no new semantics beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Actively query devices for available OTA firmware updates,' which is a specific verb+resource combination. It also distinguishes itself from sibling tools by contrasting with the passive z2m_health_report and by mentioning follow-up with z2m_ota_update.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance: 'Prefer z2m_health_report for a passive answer' and 'use this only when you need a fresh check.' It also tells the agent to follow with z2m_ota_update to actually flash, giving clear alternatives and when-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_configure_deviceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds significant behavioral context beyond annotations: the battery wake requirement and the timeout failure mode. The description aligns with annotations (readOnlyHint=false, idempotentHint=true) without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: defining the action, specifying when to use, and pointing to alternatives. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter and no output schema, the description covers purpose, usage trigger, prerequisites, and alternatives. It is complete for an agent to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the 'device' parameter, with a clear description in the schema. The tool description does not add extra parameter-level detail, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool re-runs a device's configuration routine, specifically bindings and attribute reporting. It also distinguishes from sibling tools by naming z2m_interview_device and z2m_configure_reporting as alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use when a device stopped reporting values.' It also gives exclusions and alternatives, plus a critical prerequisite for battery devices. This leaves no ambiguity about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_configure_reportingA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds significant context beyond annotations: battery devices must be woken immediately, not all devices support the command, and setting maximum_report_interval to 65535 disables reporting. It also explains the trade-off between responsiveness and battery life. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five sentences, each delivering specific value: purpose, motivation, special value, device caveat, and alternatives. Front-loaded with the core verb and resource, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers key operational constraints (battery wake-up, device support, alternatives) and the schema documents all parameters. With no output schema, the description does not need to explain return values. It is complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all parameters described, so baseline is 3. The description does not add meaning beyond the schema—it merely reiterates the 65535 behavior already in the schema's maximum_report_interval description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Configure how often a device reports an attribute.' It also distinguishes itself from sibling tool z2m_configure_device by explicitly mentioning 'one attribute' versus 'whole default configuration'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly names alternatives: 'Use z2m_get_device to see existing configured_reportings, and z2m_configure_device to re-run the whole default configuration instead of one attribute.' Also provides practical when-to-use guidance with battery wake-up and device support caveats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_interview_deviceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false), the description discloses that the device may be briefly unavailable and that battery devices must be awake. It also details what internally happens (re-reads endpoints, clusters, basic attributes), adding valuable behavioral context not present in structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, then caveats, then use cases. Every sentence earns its place with no redundancy or filler, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description covers purpose, behavioral effects, prerequisites, alternatives, and a recovery scenario. Combined with the annotations, it gives an agent everything necessary to decide when and how to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single parameter 'device' with a clear description ('Device friendly_name or ieee_address.'), so schema coverage is 100%. The description does not add parameter-specific semantics beyond what the schema provides, but the usage guidance (check interview_state) is helpful context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 're-interview' with a clear resource 'device', explains the action (re-reads endpoints, clusters, basic attributes), and explicitly distinguishes from sibling tool z2m_configure_device. It is unmistakable what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit when-to-use scenarios (firmware upgrade, stuck interview), a prerequisite (check interview_state via z2m_get_device), and an alternative (z2m_configure_device when reporting stopped). This is clear guidance on selection and context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_list_groupsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint: false. The description adds context by stating the return format ('Returns {total, groups[]}') and the meaning of an empty list, which enriches the agent's understanding beyond the annotations. No contradictions found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each with a distinct purpose: what it does, when to use alternatives, and what it returns. It is front-loaded with the primary function and contains no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and rich annotations, the description fully covers the tool's behavior. It explains the return structure and the 'empty list' case, and provides clear sibling differentiation, making it complete for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is 100% (vacuously). Baseline for 0 params is 4. The description adds no parameter-specific details, but none are needed; it appropriately focuses on output semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List Zigbee groups with their members and scenes.' It uses a specific verb and resource, and distinguishes itself from siblings by directing to z2m_manage_group for group changes and z2m_list_devices for individual devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use z2m_manage_group to change groups, and z2m_list_devices for individual devices.' This clearly indicates when to use this tool versus the alternatives, covering both modification and device-level queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_network_mapA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the read-only and idempotent annotations, the description discloses non-obvious behavioral traits: the scan reduces network responsiveness and can take 10 seconds to several minutes. It also clarifies the return shape, which is not in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, performance warning, and usage guidance. Front-loaded with the core action, each sentence adds distinct value, and the structure is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description outlines the return structure and performance characteristics. It covers the main usage caveats and alternatives, making it comprehensive for a complex scanning tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds only a minor reference to 'unless raw is set,' which duplicates the schema's raw parameter description. No significant new parameter semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (scan mesh topology) and the specific outputs (per-device parent, depth, link quality, route count, orphaned/weak devices). It distinguishes itself from siblings by naming z2m_health_report and z2m_list_devices as alternatives for link quality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (deliberately, not routinely) and what to prefer instead (z2m_health_report or z2m_list_devices for per-device link quality). Also includes a warning about performance impact, providing clear context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_rename_deviceA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that renaming changes the MQTT topic, breaks automations/dashboards/scripts until updated, and is reversible only by renaming back. This adds significant context beyond the readOnlyHint=false annotation, which only signals a write operation without specifying side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, side effects/reversibility, and alternative tool. Every sentence earns its place with no repetition of annotations or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no output schema, the description covers the action, consequences, reversibility, and parameter behavior. Combined with comprehensive schema and annotations, the agent has enough information to invoke it correctly without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (device, new_name, homeassistant_rename all documented). The description reinforces the folder-structure behavior already in the schema and adds MQTT-topic impact context, but it does not provide new parameter-level detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with 'Rename a device's friendly_name' – a specific verb and resource. It clearly distinguishes itself from z2m_set_device_options by noting that this tool affects the MQTT topic, while the alternative handles behavior changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use z2m_set_device_options for behaviour changes that do not affect the topic,' giving a direct alternative and exclusion. Also warns about breaking references, helping the agent decide when renaming is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_set_stateA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations, the description discloses critical behaviors: the command is fire-and-forget (publication without waiting), success does not mean the device acted, and reversibility. It also clarifies that mode='get' does not change anything. This adds significant context absent from annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a bit dense but every sentence adds relevant information. It is front-loaded with the primary action and contains no fluff, though it could potentially be tightened while retaining the essential caveats.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, it effectively explains the outcome semantics (message sent vs device acted) and directs to z2m_get_device for confirmation. It covers all parameters and key behavioral nuances, making it sufficiently complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the effect of mode='get' (no change) and providing additional payload examples, enriching the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as sending a state command to a device or group, with concrete payload examples. It distinguishes from sibling read tools by noting mode='get' only requests a value, and from z2m_get_device which is for reading state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use mode='get' vs 'set', and directs users to z2m_get_device to confirm device action and consult valid properties, naming the sibling tool as the alternative for reading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_get_logsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint etc.), the description reveals that logs are only captured while the server is connected, meaning history may be empty on fresh start. It also explains the default buffered behavior and how watch_seconds changes collection mode. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a purpose: definition, behavior explanation, and usage guidance. It is front-loaded with the core action and efficiently packs essential context without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a read-only log tool: it explains behavior, edge cases (fresh start), and recommended use cases. Sibling tools and annotations provide additional context, and the absence of an output schema is acceptable here since the tool's return values are self-explanatory log lines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already well-documented. The description adds value by clarifying the default mode (buffered history) and the relationship between watch_seconds and the default behavior, enhancing understanding beyond the schema. It also hints at the level parameter's limitation (no debug lines) which is echoed in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads Zigbee2MQTT bridge logs and lifecycle events, listing specific event types. It distinguishes itself from siblings like z2m_health_report by focusing on log retrieval rather than device status. The verb 'Read' and resource 'bridge logs' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use for diagnosing pairing failures and rejoin loops, and recommends an alternative (z2m_health_report) for finding affected devices. This provides clear when-to-use and when-not-to-use guidance, plus an alternative tool suggestion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_permit_joinA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses an important side effect beyond the annotations: 'While open, any nearby Zigbee device may join, so keep the window short.' This adds behavioral context not captured by readOnlyHint or destructiveHint. It also explains how to close the network and the purpose of router scoping, plus the idempotent nature of the time parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the core action, followed by a safety warning and a pointer to an alternative. Every sentence earns its place with no redundancy. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for the tool's complexity: it covers what it does, when to use it, safety considerations, how to close it, optional scoping, and how to check current state. With no output schema, the description compensates by providing all necessary context. The schema fills in parameter specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full descriptions for both parameters (time and device). The description elaborates further by advising to use 0 to close, and by recommending router scoping as 'the recommended way to pair a device into a specific part of the mesh.' This adds contextual meaning beyond the schema, though the schema already covers the basics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Open or close the network for new devices to join.' It uses a specific verb and resource, and distinguishes itself from sibling tools like z2m_bridge_info (which checks status) and device-specific tools. The mention of scoping to a single router adds clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool ('Open or close the network'), gives best practices ('keep the window short and close it with time=0 when finished'), and names an alternative for checking state ('Use z2m_bridge_info to see whether joining is currently open'). This provides clear guidance on both usage and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• z2m_set_device_optionsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate mutation and idempotence, but the description adds crucial behavioral detail: options are merged (not replaced), persist in the Zigbee2MQTT configuration, and the response reports whether a bridge restart is required. This goes well beyond annotation hints and provides meaningful operational transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each carrying essential information: purpose, merge/persistence behavior, prerequisite lookup, and alternative tool guidance. No fluff, perfectly front-loaded, and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description covers all necessary context: prerequisites, alternative tools, merge semantics, persistence, and return-value meaning (restart required). For a mutation tool with two parameters, this is thorough and self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers both parameters (device and options) with 100% coverage. The description enriches this by providing a concrete JSON example for the options object and explicitly clarifying the merge semantics, which adds practical meaning beyond the schema's field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Change Zigbee2MQTT device options' and lists example option names (transition, retain, debounce, etc.), clearly defining a specific verb+resource. It also distinguishes itself from z2m_set_state and z2m_set_bridge_options, making the unique purpose immediately apparent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs users to call z2m_get_device first to inspect settable_options, and explains when to use z2m_set_state (change what a device is doing) and z2m_set_bridge_options (bridge-wide settings). This provides clear, actionable usage guidance and differentiates the tool from its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.