Server Quality Checklist

Tool Scores

• remove_nameC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It only says 'remove', which implies mutation but does not explain consequences, error handling, permissions, or whether the operation is reversible. This is a significant gap for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no unnecessary words or repetition. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, no annotations, and incomplete parameter descriptions. The description is too sparse to guide an agent reliably; it lacks return value information, error scenarios, and any operational context. The complexity is low, but the description does not compensate for the missing structured information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% for both parameters. The description mentions 'device' and 'alias', which hints that 'id' likely refers to a device identifier and 'name' to the alias, but it does not explicitly define either parameter. The description adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Remove an alias from a device' uses a specific verb ('remove') and resource ('alias from a device'), clearly distinguishing it from sibling tools like set_name. It unambiguously states the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no when-to-use context or alternatives. It only states what the tool does, leaving the agent to infer usage conditions. There is no mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_deviceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral context. It explains brightness scale and temperature unit, but does not disclose that null parameters are likely left unchanged, nor any prerequisites or side effects of the mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and immediately clarifying ranges. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does not state what the tool returns or any error behavior. It also omits details about omitted parameters and does not explain the 'id' parameter, leaving gaps for a tool with only three parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides parameter names and types, but the description adds semantic meaning for brightness (range and off value) and temperature (Kelvin). The 'id' parameter remains undocumented, leaving its purpose ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Control a device' and then specifies brightness and temperature ranges, making it clear the tool adjusts these two properties. The specific units and ranges distinguish it from sibling tools like set_mired or set_level, though the verb 'control' is somewhat generic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given for when to use this tool over alternatives such as set_level, set_mired, or toggle. The description implies usage through parameter names but lacks context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_bridgeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits on its own. It only states the action, but does not mention whether removal is irreversible, what side effects occur, or whether special permissions are needed. The term 'Remove' implies destructiveness, but no specifics are given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler or unnecessary detail. It is concise and front-loaded, stating the action and object efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple removal tool with only two parameters and no output schema, the description states the core purpose but omits behavioral details and usage conditions. It is minimally adequate but leaves gaps regarding expected behavior and context, especially given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the ip and port parameters. While the parameter names are self-explanatory, the description adds no semantic meaning beyond what the schema already provides, failing to compensate for the lack of coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Remove' and identifies the resource as 'a registered logical bridge', clearly distinguishing this from sibling tools such as add_bridge. It communicates the action and target without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool relative to alternatives. There is no mention of prerequisites, when removal is appropriate, or when another tool (e.g., remove_name) might be more suitable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metadataB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It provides minimal behavioral context: 'HTTP-only on the REST side' is a transport detail, not a behavioral trait like side effects or authorization requirements. It does not state whether the operation is read-only, what the response format is, or any rate-limiting/pagination behavior. This leaves significant gaps for an agent to safely invoke the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence (about 20 words) that conveys the core resource, its content, and an interface constraint. There is no filler or redundancy; every phrase adds value. It is well-structured and front-loaded with the key term 'metadata'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with no parameters and no output schema, the description adequately explains what data is returned (capabilities + states per device). However, it omits practical details an agent might need: whether the response is a list/map, if authentication is required, and whether this is a read-only operation. The lack of annotations and output schema shifts more responsibility onto the description, which it only partially fulfills.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already reflects that with 100% coverage. There is nothing for the description to add about parameters. Per the rubric, 0 params earns a baseline of 4, and the description does not need to compensate further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource as 'Declarative bridge metadata (capabilities + states per device)', which clearly distinguishes it from sibling tools like get_devices or get_status. The verb is implicit in the tool name 'get_metadata', but the description clarifies what is retrieved. Slight deduction for not explicitly stating 'returns' or 'retrieves', though the meaning is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'as consumed by federation peers' implies this tool is intended for federation integration rather than general device queries. However, there is no explicit guidance on when to use it over alternatives, nor any mention of exclusions or preferred scenarios. The usage context is only partially specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_nameB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral burden. It states that aliases are display-only, which is helpful, but it does not disclose whether the operation overwrites existing names, requires permissions, has side effects, or what the response looks like. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the core action, and has zero redundancy. Every word adds value, making it exceptionally concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations, no output schema, and only 0% parameter coverage. The description explains the core purpose but fails to cover necessary context like success behavior, effects on existing data, error conditions, or relation to other tools. This makes it incomplete for an agent to invoke confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It implies that 'name' is the display alias and 'id' identifies the device, but it does not explain any constraints, formats, or uniqueness. This partially adds meaning beyond the raw schema but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (assign), the resource (device), and the specific concept (display alias). It also distinguishes itself by clarifying that aliases are for display only and not for ID resolution, which differentiates it from naming that affects identity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when you need a display-only label) but does not explicitly mention alternatives or exclusions. For example, it doesn't reference sibling tools like set_device or remove_name, nor does it say when to avoid this tool (e.g., for ID resolution).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• toggleB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly discloses the state-flipping behavior and the full-brightness action, which is core. However, it omits details like what happens with an invalid id, whether it returns the new state, or any side effects like state persistence. For a simple tool, the disclosed behavior is reasonably transparent but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, immediately front-loading the action and providing precise conditions. Every word earns its place; there is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple toggle tool, the description covers the core state transition but lacks any mention of return values, error handling, or prerequisites (e.g., device existence, connectivity). Given the absence of an output schema and annotations, the description is somewhat incomplete but adequate for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description does not mention the 'id' parameter at all. Although the tool name implies 'id' is the device identifier, the description fails to explicitly connect the parameter to the toggle action, leaving the user to infer its meaning. The description does not compensate for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Toggle a device on or off' with specific behavioral details ('If on, turns off. If off, turns on at full brightness'). This distinguishes it from sibling tools like set_device or set_level by indicating the specific toggle behavior and the full-brightness default.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is given. The description does not mention alternatives or scenarios where toggle is preferred over set_device or set_level. The context of toggling is implied but not stated for usage decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• register_deviceA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses a useful security/behavioral trait—pairing code must be sent in the body, never the URL—and mentions optional IP/name. However, it does not describe potential side effects, network requirements, failure modes, or return behavior, leaving an incomplete picture for a mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundant phrasing, and front-loaded with the action. The body-vs-URL note is valuable and concise. Every clause contributes to the tool's core usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters, no annotations, and no output schema, the description covers the essential inputs but omits contextual details such as device prerequisites, expected behavior if commissioning fails, and return values. It is adequate for a simple tool but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent (0% coverage). The description adds that 'code' is the pairing code and that 'ip' and 'name' are optional, giving some meaning beyond the bare property names. Yet it does not explain IP format, name purpose, or constraints, so compensation is only partial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool commissions a new Matter device using its pairing code, with optional IP and name. This distinguishes it from sibling tools like add_bridge or unregister_node by specifying the resource (new Matter device) and the method (commission using pairing code).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as add_bridge or get_devices. It does not explain prerequisites, target device state, or scenarios where this is preferred, leaving the agent without exclusions or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_miredA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the action and does not mention permissions, reversibility, side effects, or return values. For a mutation tool, this lack of context is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and includes the key resource and target. There is no filler or repetition, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two parameters, but the description lacks context about valid mired values, what types of devices are supported, or any behavioral outcomes. Given the absence of annotations and output schema, the description is minimally acceptable but leaves several gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only 'id' and 'mireds' with minimal titles, and the description adds meaning by explaining that 'mireds' refers to color temperature and 'device' implies the id is a device identifier. However, it does not define mireds as a unit or clarify the id format, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'set' and the resource 'color temperature in mireds' for a device. This distinguishes it from sibling set tools like set_level or set_ac, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool is used to set color temperature, but it does not explicitly state when to use it versus alternatives like set_level or toggle. No exclusions or alternative suggestions are provided, leaving the usage guidance implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_controlA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only gives an example payload. It does not disclose effects on devices, failure modes, authorization requirements, or whether actions are applied atomically, leaving the agent without critical behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with an inline example, front-loading the core purpose and format. There is no wasted wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and a minimal schema, this short description is insufficient for a batch mutation tool. It omits details like error handling, partial success behavior, validation rules, and the full set of supported action properties, despite an output schema existing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero schema description coverage, the description provides the only semantics: actions contain id, brightness, and temperature. However, it does not clarify which fields are required, value ranges, or whether additional properties are accepted, leaving notable gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as controlling multiple devices simultaneously, with an example action format. This distinguishes it from single-device siblings like set_device, toggle, and set_level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Control multiple devices at once' clearly implies a batch usage context and distinguishes this from single-device tools. However, it does not explicitly mention alternatives or exclusions, so it stops short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_levelA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the basic operation and range, without disclosing side effects, permissions, or behavior under certain conditions. Minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundancy or filler. It conveys the core action and key constraint efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple but lacks behavioral details, output schema, and annotations. The description is enough for a basic setter but omits when-to-use context and potential side effects, making it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds the 0-254 range for 'level', which helps. However, 'id' is left unexplained, leaving a gap. It partially compensates for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('set'), resource ('brightness level'), and target ('device'), with a clear range (0-254). This distinguishes it from sibling tools like set_mired and set_device.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use case is clear: setting brightness. However, it does not explicitly mention alternatives or when not to use it. The context is sufficiently clear to meet the 'clear context' criterion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sensorA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only restates the action without revealing any potential side effects, error conditions, access requirements, or return behavior. This is minimal and insufficient for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence containing only the essential information. No filler or redundant content, and the key point (single sensor, device ID) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (one parameter, no output schema), the description covers the core operation. However, with no annotations, no output schema, and no usage guidance, it leaves out important context such as return format or expected behavior, making it merely adequate rather than fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description adds meaning by clarifying that the 'id' parameter refers to a device ID. Although the phrasing 'by device ID' could be clearer (e.g., whether it is the sensor's own ID or a related device ID), it still provides context beyond the bare schema property name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource: 'Get a single sensor's data by device ID.' It specifies 'single sensor' which distinguishes it from the sibling tool 'get_sensors' (plural). The phrasing is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving one sensor via its device ID, but it does not explicitly contrast with 'get_sensors' or provide any when-not-to-use guidance. There is no mention of alternatives, making the guidance only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unregister_nodeA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full behavioral disclosure. It mentions 'Unpair' and 'clean up' but does not state side effects, reversibility, failure conditions, or whether it only affects phantom entries. This leaves significant ambiguity for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the action, target, and purpose in under 15 words. Every part earns its place without wasted verbiage, making it highly effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple unpair operation with one parameter, the description covers the core purpose and use case. However, since there are no annotations or output schema, it lacks details about return values, error handling, and safety implications, leaving some gap for an agent to fully understand the operation's consequences.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'by node_id', connecting the parameter to the action, which adds meaning beyond the schema's type and required flag. However, with 0% schema description coverage, the description does not fully compensate by explaining how to obtain node_id or any constraints, though the single-parameter simplicity mitigates the gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Unpair a fabric node') and specifies the resource and scope ('by node_id (clean up phantom/duplicate entries)'). It distinguishes from siblings like 'register_device' and 'remove_bridge' by focusing on unpairing nodes and cleaning up stale entries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'clean up phantom/duplicate entries' provides clear context for when to use this tool, indicating it is meant for removing invalid or duplicate node registrations. However, it does not explicitly exclude alternatives or mention when not to use it, such as for active nodes or bridges.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_devicesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. The verb 'List' implies a read-only operation, which is a useful safety signal. It also states the output includes 'current states and aliases'. However, it doesn't disclose pagination, rate limits, or permission requirements, and there's no explicit statement that it's non-destructive. This is adequate but has gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence. Every word contributes: 'all devices' specifies scope, 'physical and logical' clarifies the taxonomy, 'current states and aliases' previews the return content. No redundant or filler words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no parameters and an output schema (per context signals), the description needs only to state the tool's purpose and key output characteristics. It does so effectively, mentioning the scope and included fields. This is a complete description for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description adds no parameter-specific details. The baseline of 4 applies, as no parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and the resource 'devices', further specifying 'physical and logical' and 'current states and aliases'. This clearly distinguishes it from sibling tools like get_lights or get_sensor, which target specific device categories. However, it doesn't explicitly name alternatives, so it's not a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you need an overview of all devices, as indicated by 'List all devices'. It doesn't provide explicit when-to-use vs alternatives, nor does it mention any exclusions or prerequisites. This is minimal guidance, so a score of 3.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lightsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It implies a read-only operation ('List') and specifies output units, but it does not disclose potential side effects, authentication requirements, or whether the data is cached or fresh. This is acceptable but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the action and resource, then specifies key output details. There is no redundancy or unnecessary text, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 0-parameter, read-only list tool with an output schema, the description is largely complete. It clearly states what the tool returns, and the output schema can cover detailed fields. Slight lack of usage guidance is the only gap, but not critical here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides no parameter details. The description adds meaningful context about the returned data (normalized brightness and Kelvin), which compensates for the absence of parameters and meets the baseline for a 0-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('lighting devices'), clearly distinguishing it from sibling tools like get_sensor or get_devices. It also adds detail on the output (normalized brightness and color temperature), making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With many sibling get_* tools and no exclusions or context clues, the agent is left to infer the appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_bridgeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does disclose a security-relevant behavior: api_key is sent in the request body, never the URL. However, it does not mention other behavioral traits such as idempotency, response format, or prerequisites for successful registration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and the second sentence adds a necessary nuance about api_key. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 3-parameter tool with no output schema and no annotations, the description covers the purpose and the meaning of all parameters, including an important security consideration. It omits return values and error scenarios, but these are not critical for this tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by attaching meaning to each parameter: ip and port are the bridge's address, and api_key is an optional credential for authenticated peers. This goes beyond the raw type definitions in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Register' and clearly names the resource ('remote logical bridge') and key connection details (IP and port). This distinguishes it from sibling tools like remove_bridge and register_device.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the purpose ('Register a remote logical bridge'), but the description does not explicitly state when to use this tool versus alternatives like remove_bridge, nor does it provide exclusions or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_acsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. The verb 'List' implies a non-destructive read operation, and the specific data fields are disclosed. However, it does not address potential behaviors like pagination, authorization requirements, or edge cases (e.g., no ACs found), which would add transparency but are not critical for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and includes necessary clarification in parentheses. Every word contributes value, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with an output schema, the description provides the essential purpose and data fields. It does not mention pagination or failure modes, but these are not strictly necessary given the simplicity and the presence of an output schema that defines return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already covers them (100% coverage). The description adds no parameter-specific information, which is appropriate. The baseline of 4 applies since parameters are absent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'List all ACs'. It further clarifies that ACs are Matter Thermostat endpoints and enumerates the data fields returned, making it distinct from siblings like get_ac or set_ac.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a listing operation for all ACs but does not explicitly mention when to use this tool versus get_ac (single AC) or set_ac. No alternatives are referenced, so the guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refreshA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the operation is a forced refresh, implying it may be resource-intensive or override caching. However, it does not mention side effects like whether states are guaranteed to change, if the operation is blocking, or any required permissions or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that starts with the action ('Force refresh') and immediately specifies the target. There is no redundant or unnecessary wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter tool, the description is fairly complete: it identifies the operation, scope, and forcefulness. However, it lacks any mention of return values or potential side effects, which would be helpful given there is no output schema. Still, the description is adequate for a straightforward refresh action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is trivially 100%. The description adds meaning by clarifying the exact scope of the refresh (Matter bridge and logical bridges), which is necessary given there are no parameters to configure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Force refresh') and the scope ('all device states from Matter bridge and logical bridges'). This is specific and distinguishes it from sibling get/set/control tools, which are about querying or modifying individual devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is implied: use this when you need to force a refresh of all device states. However, no explicit alternatives or exclusions are mentioned, such as when to use get_sensor instead of refreshing everything. The guidance is not misleading, but it lacks direct comparison to other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It specifies that the output includes counts of lights on/off, active sensors, connected bridges, and total devices, which gives useful behavioral information. However, it does not disclose response format, potential errors, or whether the data is live or cached. The description is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single well-structured sentence with the key phrase 'Quick summary' front-loaded. It wastes no words and clearly enumerates the contents of the summary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description adequately conveys the tool's purpose and what it returns. The list of counts (lights on/off, active sensors, connected bridges, total devices) forms a complete picture for an AI agent to decide when to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain. The schema is empty and the description adds no parameter info, but the baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a 'Quick summary' of counts for lights, sensors, bridges, and devices. This distinguishes it from sibling tools like get_lights and get_sensors, which likely return detailed individual records. The verb 'summary' plus resource categories makes purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Quick summary' implies this is for high-level overviews, but the description does not explicitly state when to use it over alternative getter tools or exclude specific scenarios. There is no mention of alternatives or when-not-to-use, so the usage guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_acA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a read operation ('Get') but does not describe the response format, error handling, or side effects. For a simple getter, this is some but not extensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action and the parameter. It contains no wasted words and is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is low complexity with a single parameter and no output schema. The description tells what the tool does and the required input, but does not explain what 'state' includes. Given the simplicity, the description is mostly complete, with only a minor gap in return details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines an 'id' string with no description (0% coverage). The description clarifies that the id parameter is a 'device ID', providing crucial semantic meaning beyond the schema. This effectively compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and identifies the resource as 'a single AC's state', with the qualifier 'by device ID'. This clearly distinguishes it from sibling tools like list_acs and set_ac, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool when you need a single AC's state by device ID. It does not explicitly name alternatives or exclusions, but the intent is evident given the sibling list_acs and set_ac tools, so the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_climateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well: it discloses that only devices reporting temperature/humidity are included, that data comes from thermostat local_temperature and standalone sensors, and that each entry has a 'kind' field. It does not cover error handling or rate limits, but for a simple read tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary action, and contains no filler. Every clause adds value: scope, device types, optional filtering, and the output 'kind' field are all relevant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (so return fields need not be enumerated), the description provides complete context: it explains what values are read, which devices are included, how to filter with 'id', and the meaning of 'kind'. It adequately differentiates the tool's purpose within a large sibling set.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only contains 'id' with no description, so the description is the sole source of parameter meaning. It explicitly states 'If id is given, return one device only', fully explaining the behavioral effect. Since there is only one parameter and it is well explained, this warrants a top score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Read' and the resource 'temperature (°C) and/or humidity (%)' for relevant devices, distinguishing it from siblings like get_sensor or get_sensors by specifying device types (thermostats and standalone temp/humidity sensors) and the 'kind' field. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is implied rather than explicit. The description notes that without an id it reads for every reporting device and with an id it returns one device, giving clear conditional usage. However, there is no direct comparison with alternatives or explicit 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sensorsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. 'List' clearly indicates a read-only operation, and the enumerated metric types add meaningful context about the returned data. However, it does not mention pagination, authentication, or error behavior, so it is not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the verb 'List' and immediately follows with the object. It contains no filler or redundant words, and every element (resource and metric examples) earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no annotations) and the presence of an output schema, the description sufficiently covers the core purpose and expected data content. It does not need to explain return values since the output schema exists. It could slightly improve by clarifying whether all sensors are returned or if there is any default filtering, but overall it's complete for a basic listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is trivially complete with 100% coverage. The description does not need to compensate for missing parameter documentation. The baseline for 0-param tools is 4, and the description adds no unnecessary parameter-related noise.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' with a clear resource 'sensor devices' and explicitly enumerates the metrics (illuminance, temperature, humidity, occupancy, etc.). This clearly distinguishes it from siblings like 'get_sensor' (singular) and 'get_devices' by narrowing to sensors and their metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context (list all sensors and their metrics) but does not explicitly mention alternative tools or exclusions. The plural 'sensor devices' and inclusion of metrics imply this is the go-to for broad sensor listings, but no explicit 'when-to-use' is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_identitiesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully shoulders the behavioral disclosure burden. It specifies that physical Matter devices return the full set while logical devices report only id + names, and it explains that the id is a one-way hash, making this tool the only way back to node_id/endpoint_id. This level of detail goes well beyond a minimal description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and information-dense. It front-loads the main purpose, lists attributes, then covers the use case and id behavior. Every sentence contributes value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description correctly avoids repeating return field structure. It covers the tool's main behavior, the physical vs. logical device distinction, and the special id decoding behavior. For a single-parameter, read-only tool, this is complete enough for proper selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single optional id property with no description (0% coverage), so the description must compensate. It does so excellently by explaining that providing id returns just that device and decodes the hash back to its node_id/endpoint_id and identity. This is exactly the semantic meaning an agent needs to use the parameter correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns hardware identity for every device, listing specific attributes such as vendor, product/model, versions, serial number, and Matter device-type ids. It distinguishes itself by mentioning the ability to decode a dev_ id back to node_id/endpoint_id, which is not evident in sibling tools like get_devices or get_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a concrete use case: 'Use to tell apart bridged children of one hub (e.g. Aqara sensors) that changed IDs after a re-pairing.' It also explains the behavior when an id is given. However, it does not explicitly compare with alternative tools or state when not to use this tool, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_acA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and delivers: it explains side-effects of on=True (resumes last non-zero SystemMode, defaults to Cool=3), mode overriding on, routing of setpoint, and fan_speed acceptance/rejection behavior. This is rich, non-obvious behavioral disclosure that fully compensates for missing annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, using a clear lead sentence and concise bullet points. Every sentence adds information, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a control tool with 5 parameters and no output schema, the description covers all parameter semantics, crucial behavioral nuances, and default behaviors. It is complete enough for an agent to invoke the tool correctly without further documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no parameter descriptions (0% coverage), but the description explains each parameter's meaning, units, allowed values, and device-specific behavior (e.g., fan_speed forwarded vs rejected). This adds substantial value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource ('Control an AC') and enumerates the specific controllable aspects (on, mode, setpoint, fan_speed), distinguishing it from read-only sibling tools like get_ac and list_acs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool (to control AC settings) and its relationship to other tools is obvious from the sibling list, but it does not explicitly name alternatives or state when not to use it. It is clear context without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.