Server Quality Checklist

Tool Scores

• get_stateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a 'Get' operation, implying read-only behavior, but doesn't clarify if it requires authentication, has rate limits, returns error states, or what the output format might be (e.g., JSON with state attributes). For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Get the current state'). There is zero waste—every word contributes directly to understanding the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, 100% schema coverage) but lack of annotations and output schema, the description is incomplete. It doesn't address what the tool returns (e.g., state value, attributes, timestamps) or potential errors, which is critical for a read operation in a home automation context. The description alone is insufficient for full agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'entity_id' parameter fully documented in the schema. The description adds no additional meaning beyond what the schema provides (e.g., no examples beyond the schema's 'light.living_room', no context on entity ID formats). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('current state of a Home Assistant entity'), making the purpose immediately understandable. It doesn't explicitly differentiate from siblings like 'list_entities' (which likely lists entities rather than getting state) or 'toggle_entity' (which changes state), but the specificity of 'current state' provides some implicit distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_entities' or 'toggle_entity'. It doesn't mention prerequisites (e.g., needing a valid entity ID) or exclusions (e.g., not for modifying state). Usage is implied by the action but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states what the tool does but doesn't describe how it behaves: no information on permissions needed, rate limits, pagination, return format, or whether it's safe/read-only. For a listing tool with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's appropriately sized for a simple tool and front-loads the core purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that likely returns a list of entities. It doesn't explain what information is returned (e.g., entity IDs, names, states) or behavioral aspects like safety. For a listing tool in this context, more detail would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'domain' documented as an optional filter. The description doesn't add any parameter details beyond what the schema provides, so it meets the baseline score of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all available entities in Home Assistant'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_state' which might also retrieve entity information, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'get_state' (which might retrieve specific entity states) or 'toggle_entity' (which modifies entities). The description implies a broad listing function but doesn't specify use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• toggle_entityC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the basic action. It doesn't disclose behavioral traits such as required permissions, whether this is a destructive/mutative operation (implied but not explicit), error handling, or side effects (e.g., triggering automations).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, front-loading the core action. It's appropriately sized for a simple toggle operation with well-documented parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a mutative tool. It lacks context on permissions, error responses, or what happens post-toggle (e.g., state change confirmation), leaving gaps for agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (entity_id with examples, state with enum). The description adds no additional meaning beyond implying the tool uses these parameters, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('toggle') and resource ('a Home Assistant entity'), specifying the action of switching between on/off states. It distinguishes from siblings like 'get_state' (read-only) and 'list_entities' (listing), but doesn't explicitly differentiate from 'trigger_automation' (which might involve entities).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., entity must be toggleable), exclusions (e.g., not for sensors), or compare to siblings like 'get_state' for checking current state before toggling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trigger_automationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'trigger' implies an action that may have side effects (e.g., starting an automation sequence), the description doesn't specify whether this requires authentication, what happens if the automation fails, or if there are rate limits. It lacks critical context for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple tool with one parameter. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a mutation operation with potential side effects), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like error handling, response format, or prerequisites, leaving significant gaps for the agent to navigate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'automation_id' fully documented in the schema. The description doesn't add any parameter details beyond what the schema provides (e.g., it doesn't explain how to find automation IDs or provide examples beyond the schema's example). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('trigger') and the resource ('a Home Assistant automation'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from its siblings (get_state, list_entities, toggle_entity) which all operate on different resources, so it doesn't fully distinguish itself in context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid automation ID), when not to use it, or how it differs from sibling tools like toggle_entity. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.