Server Quality Checklist

Tool Scores

• hubitat_get_deviceA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is well established. The description adds that it returns 'details and current attributes,' but doesn't elaborate on what the response format is (though output schema exists) or any rate-limit/auth nuances. Slight value-add over annotations but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, zero waste. The purpose statement, 'Use when' scenario, and exclusion with sibling reference are all packed efficiently. Perfectly front-loaded and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with full annotations (readOnly, idempotent, non-destructive), good usage guidance, and an output schema present, the description is quite complete. It could mention pagination or the shape of returned attributes, but the output schema presumably covers return values, so this is adequate. The brief mention of 'battery, level' gives a flavor of what attributes to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so the schema already documents the single device_id parameter. The description reinforces that device_id maps to a Maker API device for fetching a single device's state, which complements the schema. With only one parameter and full coverage, the description doesn't need to overshare; baseline 3 plus the context that the ID identifies the specific device for state retrieval justifies 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Get a single device's details and current attributes'), clearly distinguishing it as a read operation for a single device. It's not explicitly differentiated from hubitat_list_virtual_devices but does state it covers current state attributes, which helps differentiate from hubitat_list_devices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent guidance: explicitly states 'Use when' for current state (switch, level, battery, etc.) and 'Do NOT use when' for just needing IDs (alternatively naming hubitat_list_devices). This is an explicit when/when-not/alternatives pattern—exactly the gold standard for this dimension.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_list_devicesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds the Rule Machine exclusion detail, which is useful behavioral context not in annotations. It could add pagination/return-volume behavior, but the output schema covers return format. Annotations carry most of the burden, so a 3 is fair.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short, purposeful sentences with zero filler. Every sentence earns its place: return fields, when to use, when not to use, and an explicit exclusion caveat. Front-loaded with the purpose statement before usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists and covers the return shape. The description appropriately handles usage context, scope limitations (Maker API only, no Rule Machine), and sibling differentiation. Could add a note about default limit or result ordering, but for a list/discovery tool with good annotations and full schema coverage, this is essentially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters (limit, query, offset) are fully documented in the schema. The description doesn't add parameter-level detail but doesn't need to. Baseline 3 is correct when schema already documents parameters completely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource ('List devices authorized in Maker API') plus explicit return fields (id, name, label, room, capabilities summary). Distinguishes from siblings by noting it lists only Maker API-visible devices, separate from hubitat_list_virtual_devices and hubitat_get_device. The scope constraint (only what Maker API sees) is specific and helpful.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'Use when' (discovering devices or finding an ID before send_command) and 'Do NOT use when' (already know the ID, use hubitat_get_device) with a named alternative. Also explicitly excludes Rule Machine rules, clarifying what is NOT returned. This is exemplary when/when-not framing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_list_virtual_devicesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds useful context about Rule Machine having no API and the mechanism for triggering rules, which adds value beyond annotations. However, it doesn't describe the output structure or whether results include hub variables as well as switches, which would add further transparency value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact at roughly 40 words, split into a purpose statement and a compact use/when-not-to-use block. Every sentence earns its place with zero filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With readOnly/idempotent annotations, simple params, and an output schema present, the tool doesn't need extensive documentation. The description addresses purpose, usage context, and sibling differentiation well. Minor gap: it doesn't clarify whether hub variables are also returned by this tool or only by get_hub_variable, but this is a modest omission given the overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 33% — only 'query' has a description ('Optional filter'). The description itself doesn't elaborate on parameter semantics beyond what the schema provides. With low coverage, the description could do more to clarify param formats, but the params are simple (limit, query, offset) and largely self-explanatory by schema definition alone. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb+resource ('List devices that look like virtual switches/buttons') and explicitly distinguishes from the sibling tool hubitat_list_devices by noting this is for virtual devices/Rule Machine levers, not physical devices. The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use ('finding a virtual switch to drive Rule Machine') and when NOT to use ('controlling physical devices (use hubitat_list_devices)'), naming the exact alternative tool. This is exemplary usage guidance with explicit exclusions and an alternative named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_get_device_commandsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, fully covering the safety profile. The description adds the behavioral detail that this is preferred over guessing command names and is a prerequisite step before hubitat_send_command, which provides useful usage context beyond the annotations. With strong annotations, the additional guidance about being a low-level lookup/discovery operation adds value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Exceptionally concise and well-structured. Three short sections: core purpose, explicit 'use when' condition, and explicit 'do not use when' exclusion. Every sentence earns its place, no fluff or redundancy. The use/do-not-use formatting aids quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read-only list tool, this is complete. The description covers purpose, use cases, and exclusions. An output schema exists to document the return structure, so the description doesn't need to explain return values. Annotations handle the safety profile. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the only parameter (device_id) documented as 'Maker API device ID' in the schema itself. The description doesn't add parameter detail beyond what the schema provides, but the schema's description is clear enough that the agent knows what to supply. Baseline 3 is appropriate when schema handles the parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'List commands available for a device via Maker API' — a specific verb+resource combination with clear purpose. It distinguishes itself from siblings like hubitat_send_command (which executes commands) by clarifying this is a listing/discovery tool, though it doesn't explicitly contrast with all sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent explicit guidance. 'Prefer this over guessing command names' establishes a strong recommendation; 'Use when: before hubitat_send_command to confirm the command exists' gives a precise trigger condition; 'Do NOT use when: sending a known simple Switch on/off' names an explicit exclusion case. This is model Tier A guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_get_hub_variableA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover readOnly=True, idempotent=True, destructive=False, and openWorld=True. The description adds value beyond these by noting this is an admin route and firmware-sensitive, giving context about access level and stability that the annotations don't convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Compact and well-structured with clear sections. Every sentence earns its place — no filler, no redundancy. The use-when/not-use-when breakdown is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  A single-parameter read tool with a rich annotation set (readOnly, idempotent, openWorld) and an output schema. The description covers the main use case and exclusion. The admin/firmware warning adds important context. Slightly more on return format could push it to 5, but this is nearly complete for a simple read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema documents the single 'name' parameter. The description implies it's the hub variable name but adds no format/syntax details beyond the schema's own 'Hub variable name' description. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read a Hubitat hub variable' — a specific verb (read) + resource (hub variable), and distinguishes from the sibling tool hubitat_set_hub_variable and from device attributes via hubitat_get_device. The admin-route and firmware-sensitive qualifiers add useful context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'Use when' and 'Do NOT use when' sections with a named alternative (hubitat_get_device for device attributes). This explicitly differentiates from siblings and clarifies when the tool is and isn't appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_send_commandA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Effectively no annotations cover the safety profile (all false hints), so the description bears the burden. It discloses that the tool validates the command via /commands before sending, which is a useful behavioral detail. However, it doesn't disclose whether the action is reversible, error behavior when commands are invalid, or whether it sends on state. The validation disclosure is solid but some behavioral context like idempotency/reversibility is missing. Given openWorldHint=true and no useful annotation context, this earns a strong score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Compact and well-structured: one-line purpose, 'Use when' / 'Do NOT use when' sections, and a validation note. Every sentence earns its place with zero filler. Two short paragraphs are highly scannable for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Has output schema, so return values need no explanation. The tool is a generic command sender with 3 simple parameters all documented. The description covers purpose, usage scope, exclusions, and validation. Slightly more behavioral detail (idempotency, side effects) would push this to 5, but it's largely complete for a well-schematized tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 3 parameters. The description adds the endpoint path format showing how value maps to the URL, but doesn't add deeper semantics beyond the schema's field descriptions. Baseline 3 is appropriate since schema does the heavy lifting and description adds modest value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Send a generic Maker API command to a device' with the exact endpoint format /devices/{id}/{command}[/{value}]. Gives concrete examples (on/off, setLevel, lock) and distinguishes from hubitat_set_color, which is just one sibling. The verb+resource+scope is specific and clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when:' with common command types and 'Do NOT use when:' naming the specific alternative (hubitat_set_color for hex colors). Also mentions validation behavior. This is exemplary usage guidance that clearly scopes when this tool is appropriate versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_set_colorA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With annotations already covering readOnlyHint=false, destructiveHint=false, and idempotentHint=true, the safety profile is established. The description adds the valuable insight that it builds the URL-encoded JSON map with Hubitat's hex extension — a non-obvious behavioral/internal detail, and warns 'Nobody constructs this by hand correctly,' justifying why the tool exists. This exceeds the annotation baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four tight sentences with zero waste. Action first, implementation detail second, usage guidance third, exclusion last. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-param tool with 100% schema coverage, an output schema, and solid annotations, the description is nearly complete. It covers purpose, usage boundaries, and internal implementation nuance. It could mention response format, but the output schema likely compensates, so only a minor gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, covering both device_id and hex with clear descriptions. The description adds the color-format details (#RRGGBB or RRGGBB) and the hex-extension JSON construction context, which enriches understanding beyond the schema. Baseline 3 is appropriate since the schema handles parameter documentation well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: "Set a ColorControl device color from a hex string." It clearly distinguishes from siblings (hubitat_send_command handles generic commands, this handles color specifically) and even calls out the unique hex-extension formatting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-use: 'Use when: setting bulb color from #RRGGBB or RRGGBB' and explicit when-not: 'Do NOT use when: only changing level or on/off.' This gives the agent clear decision boundaries and implies alternatives for level/on-off changes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hubitat_set_hub_variableA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool is admin-route and firmware-sensitive (a meaningful caveat), and that it triggers Rule Machine subscriptions on the target variable. Annotations declare readOnlyHint=false, destructiveHint=false, and idempotentHint=true, which are all consistent with the description. It adds behavioral context about side effects (rule triggering) beyond the annotations, though it doesn't fully explain firmware sensitivity implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: one purpose statement, one caveat, one when-to-use line, and one when-not-to-use line with an alternative. Every sentence earns its place with zero filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is well-complemented by a 100% schema coverage and an output schema existing. For a 2-parameter write operation with good annotations and full parameter documentation, the description adds the critical behavioral nuances (admin route, firmware-sensitive, Rule Machine side effects). It could be slightly more complete by elaborating on what firmware-sensitivity means practically, but for the complexity level, it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both name and value have descriptions in the schema), so the baseline is 3. The description does not add semantic detail about the parameters beyond what the schema provides. It doesn't explain what format 'value' expects or whether 'name' needs to be exactly as configured. At baseline 3 since schema fully covers parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Write a Hubitat hub variable') and adds a scope/characteristic ('admin route; firmware-sensitive'). It clearly distinguishes itself from siblings by noting it triggers Rule Machine subscriptions, versus hubitat_send_command for device commands. This is a strong purpose statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use ('automating Rule Machine without a virtual switch') and explicit when-not-to-use ('sending a device command') with a named alternative (hubitat_send_command). This is exemplary usage guidance that differentiates from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.