ha-nexus-agent

by Fistacho

Overview Schema Related Servers Score Discussions

Python

Local

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.18.2
Disambiguation4/5
With 310 tools spanning many namespaces, the sheer volume increases the chance of misselection. However, each tool has a distinct purpose within its named subdomain (e.g., areas, automations, card_builder), and descriptions clearly differentiate overlapping operations like REST vs WebSocket service calls. Only minor ambiguity exists (e.g., multiple state retrieval methods).
Naming Consistency5/5
Tool names uniformly follow a `namespace_verb_noun` snake_case pattern (e.g., `areas_create_area`, `entities_turn_on`). This predictable structure makes it easy to infer functionality from the name alone, with no mixing of conventions or vague verbs.
Tool Count2/5
The server provides 310 tools, far exceeding the typical well-scoped range of 3–15. While the HA domain is broad, the tool set includes many niche and redundant capabilities (e.g., multiple service call methods, extensive card_builder utilities), suggesting insufficient pruning for a focused agent.
Completeness5/5
The server covers essentially every aspect of Home Assistant management: areas, automations, blueprints, dashboards, devices, energy, entities, ESPHome, files, git, HACS, helpers, history, integrations, labels, media, notifications, scenes, scripts, statistics, supervisor, system health, themes, todo, voice, zones, and more. No critical lifecycle operations appear missing.
Average 3.5/5 across 310 of 310 tools scored. Lowest: 1.5/5.
See the Tool Scores section below for per-tool breakdowns.
- 0 of 3 community issues answered or closed in the last 6 months
- No commit activity data available
- Last stable release on August 6, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits (e.g., permissions, side effects, idempotency) beyond the fact that it performs an update. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only one sentence, but it is under-informative. While concise, it fails to add value; the space could be used to convey essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 4 parameters, no annotations, and the lack of parameter descriptions, the description is extremely sparse. An output schema exists but does not compensate for the absence of usage and parameter context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any of the 4 parameters (scope, category_id, name, icon). The agent gains no additional meaning beyond parameter names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a category via WS `config/category_registry/update`' essentially restates the tool name without adding context about what updating a category entails or how it differs from sibling tools like labels_create_category or labels_delete_category.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as labels_create_category or labels_update_label. The description lacks any usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No behavioral details beyond the 3-word description. No annotations provided. Does not mention effects on working directory, branch safety, or whether the branch must be fully merged.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely short but at the expense of completeness. The single sentence lacks necessary context, making it under-specified rather than efficiently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description provides no context on return values, error conditions, or side effects. A checkout operation involves significant state changes left unmentioned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. Description does not explain the `branch_name` parameter's role, valid values, or constraints beyond implying it must be an existing branch.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (switch) and resource (existing branch), distinguishing it from create/commit/rollback siblings. Could be more specific about what 'switch' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like `git_git_create_branch` or `git_git_commit_all`. No context about prerequisites or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits, but it fails to do so. It does not mention side effects, required prior state (e.g., an active flow), or return behavior. The agent cannot infer what happens upon submission.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but overly terse. While it avoids waste, it sacrifices necessary detail. Every sentence should earn its place; this one does not provide enough value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of sibling tools and the complexity of options flows, the description is severely incomplete. It does not explain what an options flow step entails, how 'user_input' should be structured, or what the output schema contains.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, yet the description provides no parameter information. It does not clarify the role of 'flow_id' or the structure of 'user_input,' leaving the agent to guess their meanings.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Submit' and the resource 'a step in an options flow.' It is specific and distinguishes from sibling tools like 'integrations_submit_config_flow_step' by including 'options' in the name, though the description itself does not elaborate on the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'integrations_start_options_flow' or 'integrations_abort_options_flow.' There is no mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits (e.g., what turning off means, if it affects physical devices, reversibility). The single sentence provides none.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (4 words) but this comes at the cost of necessary details. It could be expanded while remaining efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the maturity of the protocol (Home Assistant) and the number of sibling tools, this description is insufficient for an AI agent to use correctly. It lacks any context around when to use it or what the output schema provides.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description adds no meaning to the entity_id parameter. No format, source, or example is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Turn off an entity.' clearly states the verb and resource, and it distinguishes well from siblings like entities_turn_on and entities_toggle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like entities_disable_entity or entities_toggle. The description provides no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description offers no behavioral details. It does not disclose whether deletion is permanent, if it cascades to dependent cards, or any side effects. The description fails to add value beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
While short, the description is under-specified. It sacrifices necessary detail for brevity, making it less helpful for an AI agent. A concise but informative description would be preferred.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and many related sibling tools, the description is incomplete. It lacks details on return values, error conditions, and behavioral context like permanence of deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'style_preset_id' has no description in the schema (0% coverage) and the tool description provides no additional meaning. The agent has no guidance on where to obtain the ID or what format it expects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (style preset), with no ambiguity. It distinguishes itself from sibling tools like create and update style presets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like update or list. No context on prerequisites, such as whether the preset must exist or if it can be deleted while in use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose effects (e.g., idempotency, prerequisites) or what happens on activation. Very minimal behavioral info.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Only three words, but under-specified. Fails to convey necessary information for correct invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite simple schema and output schema, description lacks key details: entity_id required, what activation entails, or expected outcomes. Incomplete for reliable use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds no meaning to the entity_id parameter. Agent cannot infer its purpose or format from description alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states it activates a scene, which is a clear verb+resource. It distinguishes from siblings like listing or deleting scenes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like automations_run_script or automations_reload_scenes. No context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description fails to disclose behavioral traits such as destructiveness, idempotency, permissions, or side effects. The description is too minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, achieving conciseness, but it omits critical information, making it inadequate for a tool with no annotations and an incomplete schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two parameters, no annotations, and an output schema (unavailable), the description is incomplete. It lacks details on output, behavior, and usage context among many sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no semantic meaning beyond the parameter names. 'entity_id' is self-explanatory, but 'variables' remains vague without format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'run' and the resource 'script', and 'optionally with variables' adds some context. It distinguishes from sibling tools like automations_delete_script but lacks specificity on what 'run' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like automations_trigger_automation. No prerequisites, effects, or limitations mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only says 'trigger an automation' without describing side effects, permissions, or what happens when triggered (e.g., executing actions). Critical behavioral traits like idempotency or potential errors are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is efficient but errs on the side of under-specification. It could include more useful details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter and no annotations, the description should explain what the trigger does, how to specify the automation, and what the output indicates. It fails to provide complete context for reliable agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description repeats 'automation' but adds no context about the `entity_id` parameter, such as expected format (e.g., 'automation.my_automation') or how to find valid IDs. It offers zero additive value over the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'trigger' and the resource 'automation'. It conveys the core action, but it does not differentiate from sibling tools like automations_enable_automation or automations_run_script, which have similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., enabling vs triggering). There are no mentions of prerequisites, such as the automation needing to be enabled, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only says 'update', which is minimal. Missing details on whether updates are partial or full replacement, reversibility, or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and to the point, containing no unnecessary words. However, it is too brief and does not earn its place by adding value beyond the tool name and verb.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of 5 parameters (with only one required) and zero schema descriptions, the description is highly incomplete. It does not explain what the parameters do, nor does it leverage the existence of an output schema to set expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides no information about any of the five parameters, such as what 'data' or 'extends_preset_id' mean.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a style preset' clearly states the verb and resource, distinguishing it from create, delete, and list operations among siblings. However, it lacks specificity about which fields are updatable, which would make it a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like creating a new preset or deleting one. No prerequisites mentioned, such as requiring an existing preset with the given style_preset_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It implies a mutation ('assign') but does not disclose side effects (e.g., overriding previous assignment, validation failure behavior, or whether it is idempotent).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded, but minimalism sacrifices completeness. For a simple tool this may suffice, but more detail would improve clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no nested objects), the description omits return value (despite an output schema existing), error conditions, and prerequisites. Agent lacks confidence in correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage. The description adds no meaning to the parameters 'entity_id' and 'area_id' beyond their names. An agent needs to guess acceptable formats or values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'assign' and the objects 'entity' and 'area', distinguishing it from siblings like 'areas_get_area_entities' (which reads) and 'entities_list_entities'. However, it does not add specific verb+resource precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'areas_get_area_entities' or 'devices_list_devices_in_area'. No prerequisites (e.g., entity/area must exist) or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose consequences of disabling, reversibility, or permissions needed. 'Disable' implies a change but lacks detail on what happens to the entity's state or registry entry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (5 words), but at the expense of necessary detail. Front-loaded with key action, but lacks structure for completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a single required parameter, no annotations, and an existing output schema (not described), the description is severely incomplete. It fails to address behavioral impact, return value, or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%; description provides no additional meaning for the entity_id parameter beyond its name. Does not clarify format, source, or validation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Disable' and resource 'entity in the registry', making the action obvious. Sibling tools like entities_enable_entity and entities_get_entity are differentiated by name, but not explicitly in the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., entities_turn_off or entities_set_value). No prerequisites, side effects, or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description only mentions that it includes disabled/hidden entities, a minimal behavioral trait. No disclosure of read-only nature, rate limits, or other behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (one sentence) but omits crucial parameter details. Front-loads the purpose but lacks completeness for a tool with siblings.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of multiple entity-listing siblings and an output schema, the description does not explain return value shape or filtering capabilities, leaving significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'domain' has 0% schema description coverage, and the tool description does not mention it at all, failing to add any meaning or usage guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it gets a raw entity registry including disabled and hidden entities, differentiating it from other entity listing siblings that likely filter active entities. However, it could be more explicit about what 'raw' means.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus sibling tools like entities_list_entities or entities_list_exposed_entities. No context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states creation without mentioning side effects, authorization needs, or error behavior (e.g., duplicate name). This is insufficient for an agent to understand the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only one sentence, which is concise, but it lacks structure and important details. It front-loads the action but omits parameter context and usage notes, making it under-specified for a tool with multiple optional parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no schema descriptions, no annotations, and an output schema (not shown), the description is insufficient for complete understanding. An agent would lack crucial information about parameter formats, return values, and when to use this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain any parameters (name, icon, color, description). The agent receives no additional meaning beyond the bare schema definition, which lacks constraints or formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the action clearly: 'Create a new label'. It mentions the specific WebSocket endpoint, which adds technical context. It is easily distinguishable from sibling tools like 'labels_delete_label' or 'labels_update_label', but does not explain what a label represents in the system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as reusing existing labels. There is no mention of prerequisites or conditions under which creation would be inappropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits like side effects or safety, but it does not. The agent only learns that it 'presses' a button, with no indication of whether it causes state changes, requires permissions, or is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence that is concise but not well-structured. It lacks organization and could be expanded with essential details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no annotations), the description is too sparse. It does not explain the purpose of the button press or the expected outcome, leaving the agent with incomplete context for safe and effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for parameters, and the tool description adds no explanation for 'entity_id'. The agent is left with just a type 'string' and no format guidance, which is insufficient for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Press' and the resource 'a button entity', making it distinct from sibling tools like 'toggle' or 'turn_on'. However, it could be more specific about what 'button entity' means in the context of the system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not explain the difference between pressing a button and toggling or turning on an entity, leaving the agent without decision context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations and only a 5-word description, there is minimal disclosure of behavioral traits. It does not mention permissions, side effects, or WebSocket-specific considerations beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short but lacks substance. It is under-specified rather than concise, failing to provide enough information to be useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters and an output schema, the description is minimal. While the output schema exists, the description still misses contextual details about parameters and usage, making it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no meaning to the three parameters. It does not explain what 'domain', 'service', or 'data' represent or how they are used.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (call a service) and resource (HA service) via WebSocket. However, it does not distinguish from the sibling tool 'services_call_service', which likely has a similar purpose via a different protocol.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. No context on prerequisites or scenarios is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Reveals it's a websocket call via 'config/zone/create' but does not disclose destructiveness, idempotency, or error handling on duplicate zone names. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks substance. It is front-loaded with the action but misses critical details, making it under-specified rather than efficiently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters with no schema descriptions, the description is woefully incomplete. It does not explain required fields, defaults (radius, passive), or behavior of icon. Output schema exists but description doesn't bridge the gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% with no property descriptions. The description adds no information about parameters (name, lat, lon, radius, icon, passive). The mention of 'zone helper integration' is vague and not actionable for parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a zone', which is a specific verb and resource. It distinguishes from siblings like update/delete/list implicitly by the action. However, it could be more explicit about the geographic nature (lat/lon).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., zones_update_zone). No context on prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only says 'Update an existing label' without covering idempotency, permission requirements, error handling, or side effects. The agent is left unaware of what happens if the label doesn't exist or if fields are set to null.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) but lacks critical information. Conciseness is not sacrificed for completeness; rather, it is under-informative. The first sentence does not front-load essential details like required parameters or usage context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, 1 required, output schema exists but not described), the description is incomplete. It does not mention the output (updated label object), error cases, or how it differs from other label tools. Richer context is needed for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% coverage (no parameter descriptions), and the description adds no parameter-level information. It does not explain that label_id is required or how the optional fields (name, icon, color, description) modify the label. The agent must infer all semantics solely from parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing label', matching the tool name. It adds the WS endpoint, which is technical but still conveys the action. Among siblings like labels_create_label and labels_delete_label, the purpose is distinct. However, it could be more specific about what aspects of the label can be updated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. There is no mention of when to use this tool versus alternatives like labels_create_label or labels_delete_label. The description does not include prerequisites, exclusions, or context for appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the full behavioral burden. The description only states 'Create a style preset' without disclosing any side effects, such as whether this overwrites an existing preset with the same name, authentication requirements, or any destructive outcomes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and front-loaded, with two sentences. While concise, it achieves this at the expense of necessary detail for a tool with multiple parameters and a complex nested config object.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, a required complex 'data' object, and no guidance on error handling or return behavior, the description is incomplete. The output schema exists but is not referenced. Users lack sufficient context to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds only minimal semantic value for the 'data' parameter (calling it 'CSS-style configuration blob'). It provides no explanation for 'name', 'description', or 'extends_preset_id', leaving those parameters entirely to their schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'Create' and the resource 'style preset', making the primary action clear. It distinguishes from sibling tools like 'update_style_preset' or 'delete_style_preset' by the action of creation. However, the description could be more specific about what a style preset is in the broader context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. No context is provided about prerequisites, typical use cases, or when to use 'update_style_preset' instead. The description is purely operational without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It only implies a destructive action ('Uninstall') but does not describe side effects, irreversibility, or what happens to repository data. Output schema exists but is not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence. It is not verbose, but lacks any structure (e.g., bullet points) and could include more information without becoming overly long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, output schema present), the description is incomplete. It does not specify success indicators, error conditions, or confirm what happens after uninstallation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It adds minimal value by stating 'by id' for the parameter, but does not explain the format or how to obtain the repository id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Uninstall') and resource ('a HACS repository by id'). It distinguishes from sibling tools like 'hacs_install_hacs_repository' which performs the opposite operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites (e.g., the repository must be installed first). The description is silent on context and exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the action without disclosing behavioral details like increment amount, overflow behavior, or prerequisites (e.g., counter must exist). This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately conveys the action. However, it could benefit from structured guidance (e.g., bullet points) but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description might not need extensive detail, but it lacks any context about output, side effects, or appropriate use cases. The description is incomplete for a tool that requires parameter guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage in terms of required fields but 0% schema description coverage. The description adds no meaning to the entity_id parameter beyond what the name implies. With such low coverage, the description should provide context, but it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Increment a counter helper' clearly states the action (increment) and the resource (counter helper). It is specific and distinct from siblings like helpers_reset_counter, though it could explicitly contrast with that sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like helpers_reset_counter or other helpers. The description fails to provide context for appropriate usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the action without mentioning side effects, permissions, or state transitions, leaving the agent underinformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence that is direct and to the point. However, it sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 2 parameters, no annotations, and no output schema documentation, the description fails to cover essential details like valid option values or return behavior. It is insufficient for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning parameters are entirely undocumented. The description does not clarify what 'entity_id' or 'option' represent, nor that 'option' must be a valid option from the input_select's predefined list. No added value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it sets an input_select to a specific option, using a specific verb and resource. However, it doesn't differentiate from sibling helpers_set_* tools beyond the entity type, which is already implied by the tool name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like entities_set_value or other helpers_set_* tools. The description lacks any context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions 'delete,' implying a destructive action, but it discloses no behavioral traits beyond that. There are no annotations to supplement, so the description should explain consequences (e.g., cascading effects on associated labels), but it does not.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence and concise. However, it includes a technical reference to a WebSocket endpoint that may not be essential for an agent to understand the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two required parameters and no annotations, the description is too minimal. It does not cover typical context like error handling, success indicators, or prerequisites. The existence of an output schema is noted but not leveraged in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters (scope, category_id) are required but lack descriptions in the schema (0% coverage). The description does not explain what these parameters mean or how to use them, providing no added value over the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes a category, distinguishing it from sibling tools like labels_delete_label or labels_create_category. However, it lacks specificity about what type of category, but the inline code reference to the WebSocket endpoint adds a specificity that an agent might not need.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives (e.g., when to delete a category vs. a label). No context about prerequisites or conditions is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully shoulders the burden of behavioral disclosure. It only states the scope of search and the 'kind' tag, omitting behaviors like pagination, error handling, or rate limits, which are important for a search tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, making it very concise. However, it could benefit from a brief breakdown of parameters or usage hints without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of more specific search siblings and the lack of parameter descriptions, the description feels incomplete. It does not explain how results are returned or how to refine searches, leaving gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not elaborate on the 'query' or 'limit' parameters. The schema itself provides minimal info (e.g., 'limit' has a default), but the description adds no additional meaning about how to use these parameters effectively.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it searches across entities, devices, areas, and automations with results tagged by 'kind', providing a clear verb-resource pair. However, it does not explicitly differentiate from sibling tools like search_search_entities, which reduces clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus more specific search tools (e.g., search_search_entities, search_search_devices). There is no mention of prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description must disclose behavioral traits. It only states the function and value range. Missing details on required permissions, side effects, error handling, output, or constraints. With no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundancy. However, it is too brief and lacks necessary detail for a two-parameter tool with no schema descriptions. Conciseness is positive, but at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and low schema coverage, the description is incomplete. It fails to mention the output (though an output schema exists), any constraints on the position value, or how the tool integrates with other cover-related tools. More context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain both parameters. It only explains the 'position' parameter (range). The 'entity_id' parameter is not described at all, leaving the agent to infer its purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (set) and the resource (cover/blind/shutter position) with a valid range (0=closed, 100=open). While it doesn't explicitly differentiate from siblings like services_set_cover_tilt, the use of 'position' vs 'tilt' provides implicit distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of prerequisites, suitable scenarios, or exclusions. The description is purely declarative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only says 'stop' but does not disclose side effects, permission requirements, or what happens if the add-on is already stopped. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the key action. However, it could be slightly improved by removing 'running' if it's not always required, but overall it is concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is an output schema, the description does not need to detail return values. However, it omits critical context like what the slug refers to, and fails to differentiate among the many supervisor sibling tools. Incomplete for reliable agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter (slug) with 0% description coverage. The description does not define 'slug' or explain how to obtain it. No added meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('stop') and the resource ('add-on'), and implies the add-on must be running. It differentiates from sibling tools like 'start_addon' or 'install_addon', though 'restart_addon' also stops and starts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this vs. alternatives. For example, it does not state that the add-on must be running or that this stops it immediately. No mention of prerequisites or error conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as reversibility, side effects, or required permissions. The description carries full burden but offers minimal insight beyond the action itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks structure. It front-loads the core purpose but omits crucial details like required parameters, resulting in under-specification rather than efficient conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity with one required parameter and no output schema shown, the description is incomplete. It fails to explain what the entity_id is for or any return behavior, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'entity_id' with zero description coverage. The description does not mention this parameter or explain how to specify which automation to disable, providing no added semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Disable an automation,' which is a specific verb paired with a clear resource. It effectively distinguishes from sibling tools like 'automations_enable_automation' and 'automations_delete_automation'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There are no prerequisites, conditions, or exclusions mentioned, leaving the agent without contextual usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It does not disclose effects (e.g., overwriting, persistence), permissions required, or side effects. The example for 'syntax' is helpful but insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two short sentences. It is front-loaded and to the point. However, it could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal given the presence of 4 parameters, no annotations, and an output schema. It lacks information on return values (though output schema exists), prerequisites, and behavioral context, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only explains one parameter ('syntax') with an example. Other three parameters ('name', 'initial_value', 'inherits') are not described, leaving the agent to infer formats and behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Register' and the resource 'CSS custom property', with a relevant example for the 'syntax' parameter. However, it lacks broader context (e.g., which system or card builder) and does not differentiate from sibling tools like list or delete, but the purpose is still unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., when to register vs. update vs. delete). There are no prerequisities, constraints, or use-case examples mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description does not disclose behavioral traits such as whether the tool is read-only, has rate limits, or returns paginated results. The minimalist description adds little beyond the basic purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using a single sentence plus an example. The example adds clarity without unnecessary verbosity. It is appropriately front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description could be minimal, but it still fails to clarify the scope of 'style targets' or how the block_type parameter affects the output. The example only covers one case, leaving ambiguity for other block types.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate. It only indirectly mentions 'block_type' through the example but does not explain its meaning, possible values, or formatting. The example is helpful but insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists style targets for a block and provides an illustrative example. While it distinguishes from sibling list tools by focusing on 'style targets', it does not explicitly contrast with similar tools like 'card_builder_list_block_types'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'card_builder_list_style_categories' or 'card_builder_list_style_presets'. The description lacks any contextual usage recommendations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses partial update behavior but fails to mention idempotency, return value, or side effects. No annotations provided, so the description carries the full burden, yet it is insufficient for a mutation tool with 7 parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, very concise but overly minimal. While front-loaded with the main action, it sacrifices necessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists but is not shown; description should still clarify prerequisites (card must exist) and behavior. For a partial update tool with many parameters, much context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description adds no parameter details. The agent gets no help understanding the meaning of card_id, name, config, tags, categories, or skip_version_bump.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Update' and the resource 'existing card', distinguishing it from sibling tools like create_card and delete_card. The phrase 'Only the provided fields are changed' further clarifies the partial update behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., create_card for new cards, get_card for reading). No context on prerequisites or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavior. It mentions internal WS call but does not state destructive nature, permissions, or effects on other entities. Minimal behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no unnecessary words. Could be expanded with more useful info without becoming verbose, but currently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and presence of output schema (unseen), the description is minimal. It lacks details on success/failure, side effects, or prerequisites, making it incomplete for reliable agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description should explain parameters. It only states both are required without explaining what device_id and config_entry_id represent. The tool name and context give some intuition, but description adds little.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Purpose is clear: 'Remove a device from a specific config entry' uses specific verb and resource. It does not distinguish from siblings like devices_update_device, but the action is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Only mentions required parameters, no guidance on when to use this tool versus alternatives (e.g., devices_update_device, labels_remove_label_from_device). No when-not-to-use info.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits beyond stating it renames. It omits whether the rename is reversible, requires permissions, or impacts references, which is critical for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) and concise, but it lacks important details. It is not wasteful but could be more informative without increasing length significantly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a rename operation, the description is incomplete. It does not mention return values (despite an output schema), side effects, or any prerequisites. More context is needed for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no information about the parameters 'entity_id' or 'name'. It does not explain what these parameters represent or how to use them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Rename an entity in the entity registry' clearly states the verb (rename) and resource (entity), distinguishing it from sibling tools like get_entity or set_value.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. For example, it does not mention that renaming might affect automations or that entity_id remains unchanged.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the update operation but lacks details about behavioral implications. It does not mention needed permissions, potential side effects, reversibility, or whether it triggers a restart. With no annotations, the description carries full burden and fails to provide sufficient transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but does not earn its place. It is too terse and fails to add value beyond the tool name. Better structure would include parameter details or usage hints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool performs a mutation and has an output schema (undefined in description), the description is incomplete. It lacks information on return values, error states, and operational context, making it insufficient for an agent to use reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, repo_id, is not explained in the description. Schema coverage is 0%, and the description does not clarify what repo_id represents or how to obtain it. The parameter semantics are entirely missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: updating a HACS repository to its latest version. The name and description together uniquely identify the operation, and it is easily distinguished from sibling tools like installation, uninstallation, or listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives. There is no mention of prerequisites, context, or situations where this tool should be avoided. The description is purely functional without usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It only states a simple set operation but omits any details about side effects, confirmation, or error states. The brevity leaves the agent unaware of potential constraints or post-invocation behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. It is concise and front-loaded, providing the essential verb and target.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although the tool is simple (2 required params), the description lacks any mention of return values (despite an output schema existing) and does not cover edge cases or preconditions. Given the lack of annotations, more complete context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 2 parameters with 0% description coverage, and the description adds no semantic context for either 'entity_id' or 'value'. It does not explain what constitutes a valid entity_id or the expected range/format of the value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set an input_number value' clearly uses a verb ('Set') and a specific resource ('input_number value'), making the purpose immediately understandable. However, it does not differentiate among sibling tools like helpers_set_input_boolean, helpers_set_input_datetime, etc., which all have similar 'Set' patterns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives or any prerequisites. It lacks context about the appropriate scenarios for setting an input_number value.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states 'Set' which implies mutation, but does not disclose safety traits, error behavior, or side effects. Lacks depth for a write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (5 words), but sacrifices informativeness. While brevity is positive, the description is too minimal to be useful beyond stating the obvious.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, ambiguous sibling context, and an output schema whose content is unknown, the description fails to cover important context (e.g., return value, state changes, error cases). Incomplete for a mutating tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter descriptions exist. The description does not elaborate on 'entity_id' or 'value' – e.g., valid formats, allowed values, or constraints. Compensates nothing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set an input_text value.' clearly identifies the action (set) and the target resource (input_text value). It effectively distinguishes from sibling helpers like helpers_set_input_boolean or helpers_set_input_number by specifying the type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention any prerequisites, limitations, or context such as 'only for input_text helpers' or 'not for other helper types.' The agent receives no usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior, but it omits critical details like the default time window (hours parameter) and what constitutes a logbook entry. It only covers the optional entity_id filter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. However, it could be structured to front-load the hours parameter usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two parameters and an output schema, the description is incomplete. It lacks details about the hours parameter and the nature of logbook entries, leaving the agent without essential context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description should explain all parameters. However, it only mentions entity_id and omits hours entirely, failing to add meaning for a key parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves logbook entries with optional filtering by entity_id. It is distinct from sibling tools like history_get_state_history or history_get_error_log, though it could explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There are no exclusions or context on appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'delete' without mentioning side effects, dependencies, or safety considerations. The agent is left uninformed about repercussions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise, but includes a WS path that may be internal noise. Lacks structured presentation of key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter and output schema, the description fails to provide sufficient context: what constitutes a label, how to safely delete, expected response. Agent lacks complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description adds zero information about the 'label_id' parameter (type, source, format). Agent cannot derive how to obtain this identifier.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (delete) and the resource (label), with the internal WS endpoint. Distinguishes from sibling tools like 'labels_delete_category' and 'labels_create_label'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no prerequisites or conditions described. Agent receives no context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the action but does not mention side effects (e.g., triggering automations), permissions, synchronicity, or error conditions. Extremely minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (4 words), but conciseness sacrifices critical information. While front-loaded, it does not earn its brevity as it omits essential details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (fire an event), additional context like expected payload format or return value (output schema exists) is missing. The description is incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides no explanation of 'event_type' or 'event_data' parameters. The schema only gives types; description adds zero semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fire a Home Assistant event' uses a specific verb ('Fire') and clearly identifies the resource ('a Home Assistant event'). It distinguishes from sibling tools like services_call_service (which calls a service) and other event-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., services_call_service). No mention of prerequisites, context, or when not to use it. The description lacks any usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description should disclose behavioral traits. It fails to mention whether seeking pauses playback, what happens with out-of-range positions, or any side effects. Only the basic action is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with one short sentence, no fluff, and front-loads the action. However, it may be too minimalist for a tool with no parameter descriptions in the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool, missing key context such as expected entity type, required media player state, and output behavior. Though an output schema exists, the description should still set expectations for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It only hints that 'position' is in seconds, but does not explain 'entity_id' or clarify that 'position' is a float. Minimal added value beyond the schema itself.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('seek') and resource ('media player') with the unit ('seconds'). It is specific enough to distinguish from siblings like 'services_media_play_pause' or 'services_set_volume', though it does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or context such as requiring the media player to be active.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully disclose behavior. It mentions creating a backup and the partial condition, but omits important traits like whether it overwrites existing backups, permission requirements, side effects, or password handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is concise and front-loaded, though slightly sparse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and no annotations, the description is minimal. It lacks information about return values, error conditions, or prerequisites. An output schema exists but is not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains that 'addons' and 'folders' enable partial backup, but does not clarify 'name' or 'password' beyond what the schema indicates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create) and resource (backup), and distinguishes between full and partial backups based on parameters. It effectively separates from sibling tools like supervisor_restore_backup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as supervisor_restore_backup or supervisor_delete_backup. The description only hints at partial versus full scenarios, lacking usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral context. It only states the basic action, omitting details like required auth, rate limits, error handling, or limitations on Jinja2 features. The return value is not described despite an existing output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 6-word sentence, which is concise but lacks necessary detail. It is front-loaded and has no redundancy, but it is too terse to be fully helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description does not utilize the opportunity to explain WebSocket-specific context (e.g., persistent connection state, lifecycle). It fails to address what the output represents or how errors are surfaced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not provide any parameter details beyond the schema's name 'template'. There is no explanation of what the template string should contain, expected format, or available variables. The description adds no semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Render a Jinja2 template via WebSocket', which specifies the verb (render), resource (Jinja2 template), and medium (WebSocket). This distinguishes it from the sibling 'services_render_template' which likely does synchronous rendering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The phrase 'via WebSocket' implies it's for WebSocket-based interactions, but it does not state when a WebSocket render is preferred over the synchronous service or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Lacks detail on side effects, permissions, or idempotency. No annotations to compensate; the minimal description fails to inform the agent of creation consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at one sentence, but this comes at the cost of essential information; still, no unnecessary content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description offers no insight into return values or behavior, and the minimal input parameter lacks context. Inadequate for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; the description does not explain the 'name' parameter, leaving agents to guess its purpose or constraints beyond the schema's type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new area.' clearly states the verb (create) and resource (area), distinguishing it from sibling tools like areas_delete_area or areas_control_area.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully bears the burden of behavioral disclosure. It only states what the tool does, not how it behaves (e.g., return format, permissions, limits, whether it returns entities or their states). This is insufficient for safe and effective use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no redundancy. However, it is so concise that it sacrifices necessary detail. A score of 4 reflects efficiency but notes that a bit more content would improve completeness without harming structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no nested objects) and the existence of an output schema, the description should at least clarify what 'entities' means (e.g., entity IDs, full states). It is incomplete because it omits the return type and any prerequisites, leaving ambiguity for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the parameter 'area_id' has no schema description. The tool description does not add any semantic context for this parameter (e.g., format, source, or relationship to other tools). This completely fails to compensate for the lack of schema-level documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all entities belonging to an area' is specific, using a clear verb ('Get') and resource ('entities belonging to an area'). It distinguishes itself from sibling tools like 'areas_list_devices' (only devices) and 'entities_list_entities' (all entities globally), making the tool's scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't contrast with 'entities_list_entities' (global list) or mention prerequisite steps (e.g., needing a valid area_id). This lack of usage context leaves the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It only mentions the safe extension guard, omitting other critical aspects like irreversibility, permissions required, or what happens on success/failure. This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two short sentences that front-load the core purpose and an important condition. It is efficient, though it could benefit from slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (delete operation with one parameter) and the presence of an output schema, the description still lacks completeness. It does not explain what 'safe extension' means, how to construct the path, or what the output looks like, making it insufficient for safe use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description adds no information about the 'relative_path' parameter. The user is left guessing its meaning, format, or base directory. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a config file' using a specific verb and resource. The additional safety condition about safe extensions helps differentiate it from other file operations like write or read, though it doesn't explicitly name siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating a condition for deletion (safe extension), but does not provide explicit guidance on when to use this tool versus alternatives like files_write_config_file or files_read_config_file. No when-not or alternative tool names are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations and a brief description, the tool leaves critical behavioral details unclear: required permissions, side effects (e.g., downloading files), error handling, and reversibility. The description does not disclose that installation is a write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 11 words, achieving conciseness. However, it could include a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple install tool, the description misses key details: repo_id format, version constraints, prerequisites (e.g., HACS must be installed), and output/error behavior. The presence of an output schema is not leveraged in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for the optional 'version' parameter (pinning), but provides no explanation for 'repo_id', which is required. Given 0% schema description coverage, the description should compensate but falls short.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (install), the resource (HACS repository), and the optional version pinning. It effectively differentiates from sibling tools like hacs_uninstall_hacs_repository and hacs_update_hacs_repository.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives such as hacs_add_custom_repository or hacs_update_hacs_repository. There is no mention of prerequisites or context for its use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It only states the basic action without disclosing any behavioral traits such as whether it requires permissions, whether it is reversible, or how it affects the system state. The description is insufficient for understanding side effects or constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: a single sentence of 7 words. It is front-loaded and communicates the core purpose without any filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (setting a boolean helper), the description covers the essential purpose. However, it lacks details like expected entity_id format, error handling, or return value. With an output schema present but not detailed in the description, the completeness is adequate but minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not add any meaning to the parameters 'entity_id' or 'value'. The description simply repeats the tool's function without explaining what each parameter represents or how they should be used. This leaves a significant gap in understanding parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Set') and the resource ('input_boolean'), and specifies the possible values ('true or false'). However, it does not explicitly distinguish this tool from sibling helpers like helpers_set_input_select or helpers_set_input_text, though the name itself provides differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, context, or exclusions. The description is a simple statement without any contextual advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It only states the action ('Start a timer') and provides a format hint, omitting details about side effects, authorization requirements, or timer lifecycle.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short with two distinct pieces of information. It is front-loaded, but the brevity sacrifices completeness and fails to fully explain the tool's usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, return values are not required, but the description still lacks context on the entity_id parameter and behavior when duration is null. No prerequisites or side effects are mentioned, leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds the duration format (HH:MM:SS or seconds) which is not present in the schema (0% coverage). However, it does not clarify the 'entity_id' parameter, leaving its meaning ambiguous despite being required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Start a timer', which is a specific verb and resource. It implicitly distinguishes from sibling tool 'helpers_cancel_timer' by the action of starting versus stopping, but does not explicitly name the alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives. The duration format hint offers some implicit context but fails to explain prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It only states the basic function without disclosing return format, performance implications, or edge cases (e.g., what happens if label_id is invalid).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is efficient. However, it could be slightly expanded to include more useful context without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema (not detailed) and zero parameter descriptions, the description is insufficient. It does not explain what the tool returns or how to interpret results, leaving the agent under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning to the sole parameter label_id beyond its name. No hint about expected format or values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'entities that have the given label assigned'. It distinguishes from sibling tools like labels_list_labels (which lists labels themselves) and other label operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., labels_list_devices_with_label or search_entities). No mention of prerequisites or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as permissions required, failure modes, or asynchronicity. This leaves the agent with incomplete understanding of the tool's safety and side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, which is concise, but it lacks structure (e.g., no separate parameter group). It efficiently states purpose but misses opportunities to organize information clearly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema signal, the description does not explain return values. With 3 parameters (only target partially explained) and no annotations, the description is incomplete for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It only explains the 'target' parameter's purpose and default, leaving 'message' and 'title' semantically bare. This is insufficient for proper parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it sends a Home Assistant notification, with a specific verb and resource. It distinguishes from siblings like services_notify_persistent_create by focusing on the basic notify service. However, it could be more precise about the notification channels (e.g., mobile push).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a hint about the target parameter default ('notify service name'), implying it's the main notification tool. But it lacks explicit when-to-use or when-not-to-use guidance compared to alternatives like persistent notifications.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the return format but lacks side effect information, auth requirements, or limitations (e.g., max lines, add-on must be running). As a read operation, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence that conveys the core functionality efficiently. It is concise without being verbose, though it could benefit from including the parameter explanation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's two parameters, lack of parameter descriptions in schema, and no annotations, the description is insufficient for an agent to understand how to use the tool correctly. The output schema is not shown, but the description hints at the return format, which is good but not enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for its parameters. The description does not explain what 'slug' or 'lines' mean, despite 'lines' being only implicitly mentioned as 'N'. This leaves the agent without clear parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('get the last N log lines'), the resource ('add-on'), and even mentions the return format ('{logs: ...}'). It distinguishes itself from sibling tools like supervisor_get_addon and supervisor_get_addon_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool over other related tools, nor does it mention any conditions or prerequisites for using it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility for behavioral disclosure. It only says 'Start' without clarifying if the action is idempotent, what happens if already running, or required permissions. This omission limits agent understanding of side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no extraneous text. However, it is overly minimal, sacrificing valuable information for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and a single parameter, the description is insufficient. It fails to provide context about return values, error states, or lifecycle requirements, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain the 'slug' parameter (e.g., its format or how to obtain it). The tool name is insufficient to infer parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start') and the resource ('an installed add-on'). It effectively distinguishes from sibling tools like supervisor_install_addon, supervisor_stop_addon, and supervisor_restart_addon.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., when to use start vs restart). It also lacks prerequisites or conditions for use, such as the add-on must already be installed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as side effects, authorization needs, or synchronization. It simply states the action without additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence. It is not verbose, though it sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and no annotations, the description is insufficient. It does not explain the effect of reloading, the format of the entry_id, or the output, leaving the agent with significant ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not explain the sole parameter 'entry_id'. With 0% schema description coverage, the agent has no guidance on what this parameter represents or how to format it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('reload') and the specific resource ('a specific integration config entry'). This distinguishes it from sibling reload tools like 'system_reload_all' and 'automations_reload_automations'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of context, prerequisites, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states the action without explaining side effects (e.g., impact on running scripts, state changes, permission requirements). The minimal description is insufficient for a potentially destructive reload operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one short phrase, which is efficient but lacks structural elements like a clear subject-verb-object or additional context. It is not verbose, but could be more informative without being wordy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema (unknown structure) and no annotations, the description should at least hint at the output or effect. It does not explain what 'reload' entails, leaving a significant gap for the agent to understand the tool's impact.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the baseline is 4. The description does not add parameter-level information, but none is needed. It provides no additional meaning beyond the schema, which is acceptable given no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reload all scripts') and the source ('from YAML'). It is specific enough to convey the tool's primary function, but it does not differentiate from sibling tools like automations_reload_automations or automations_reload_scenes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there any prerequisites, side effects, or exclusions mentioned. The description offers no context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden but fails to disclose error handling, permissions, overwrite behavior, or limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, but it omits essential details, making it too brief for effective use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema and three parameters, the description lacks context about return values, error conditions, or relationships to similar tools, resulting in incomplete guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; the description adds no explanation for any of the three parameters, leaving their meaning and constraints ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (upload), source (local file on nexus host), and destination (Card Builder media directory), distinguishing it from siblings like card_builder_upload_media and card_builder_upload_image_from_url.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are prerequisites or typical scenarios mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It only states the action without revealing side effects, idempotency, permissions, or other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that conveys the core purpose efficiently. However, it could be restructured to include parameter explanations without increasing length significantly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema but lack of annotations and poor parameter descriptions, the description is incomplete. It does not explain what the tool returns or how to use the parameters correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'flow_to' and gives an example (solar export), but does not explain the purpose of stat_energy_to or stat_compensation. The description adds minimal meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Add), the resource (grid return flow to the Energy Dashboard's grid source), and provides an example (solar export). It effectively distinguishes from siblings like energy_add_grid_consumption and energy_add_solar_source.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention context, prerequisites, or exclusions, leaving the agent without decision criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description bears full responsibility. It only states 'Add a solar production source' without disclosing side effects, permissions required, or idempotency. There is no contradiction with annotations, but transparency is low.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, making it very concise. However, it may be too terse; adding parameter details would improve without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given two parameters and the presence of an output schema, the description is insufficient. It does not explain what constitutes a solar production source, how to specify it, or what the tool returns, leaving significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description provides no guidance on the parameters. 'stat_energy_from' and 'config_entry_solar_forecast' are unexplained, leaving the agent to infer their meaning from names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add a solar production source to the Energy Dashboard' uses a specific verb ('Add') and resource ('solar production source'), clearly distinguishing it from siblings like 'energy_add_battery_source' or 'energy_add_grid_consumption'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not specify when to use this tool vs alternatives, nor does it mention prerequisites or context like the need for an existing Energy Dashboard setup.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description is the sole source of behavioral info. It states the tool retrieves data, implying a read-only operation, but does not explicitly confirm idempotency, permissions, or rate limits. The description is minimal and lacks disclosure of side effects (though likely none).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and concise. However, it could be slightly more structured to include key information like the return format or parameter details. It earns its place but leaves some gaps.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no annotations), the description adequately covers the core purpose. However, it lacks contextual cues such as typical use cases, prerequisites (e.g., entity must exist), or how the returned data is structured (though an output schema exists). It is minimally sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter 'entity_id' with 0% description coverage. The description does not elaborate on the parameter beyond its name. Since the parameter is self-describing (standard entity ID) and the schema is simple, the baseline score of 3 is appropriate; the description adds no extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'full state and attributes of a single entity'. It is specific enough to distinguish from many siblings like entities_list_entities (which lists all entities) and entities_get_entity_exposure (which gets exposure settings). However, it does not explicitly differentiate from other single-entity retrieval tools like ws_get_states.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not compare with entities_list_entities for batch retrieval or ws_get_states for overall state. An agent would need to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It only says 'select option', implying a mutation, but does not mention error conditions (e.g., invalid option), permission requirements, or whether the operation is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence with no unnecessary words. However, the extreme brevity sacrifices explanatory power, but it is concise in the sense of no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity, the description fails to provide essential context such as what happens when an invalid option is provided, or how this tool differs from similar helpers. The presence of an output schema is noted but not utilized in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description adds minimal value: it identifies 'option' as the value to select but does not clarify that options must match predefined choices of the entity. This leaves ambiguity for agent invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (select) and the target (option for input_select or select entity). It effectively distinguishes from sibling tools like entities_turn_on or entities_set_value.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as helpers_set_input_select or entities_set_value. The description lacks any context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for transparency, but it only states the operation as a set, without disclosing whether it requires authentication, side effects (e.g., if the value is clamped), or what happens if the entity type is incorrect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundancy. It is concise but could be structured to include more information without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity but no annotation context, the description is too brief. It does not cover typical constraints like value ranges, entity existence requirements, or return behavior (even though an output schema exists).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no further details beyond the schema types. For example, it does not explain that entity_id must be the full entity ID format, or that value might have range constraints for climate temperature.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('set value') and specifies the exact entity types it applies to ('input_number, number, or climate temperature'), effectively distinguishing it from sibling tools like entities_turn_on or helpers_set_input_number.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives, such as entities_turn_on for switches or helpers_set_input_number for input_number helpers. The description lacks context on prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description alone must disclose behavioral traits. It only states that the entity is toggled between on and off but omits important details: what happens if the entity does not support on/off states, whether the operation is idempotent, any side effects, or prerequisites like permissions. This minimal description fails to provide sufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that efficiently conveys the tool's core purpose. Every word contributes meaning, with no fluff or redundancy. It is front-loaded and easily scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no output schema shown), the description lacks completeness. It does not mention that the entity must support on/off states, that the entity_id is required, or what the return value looks like. The agent is left guessing about preconditions and outcomes, making the description insufficient for informed invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has a single required parameter 'entity_id' with no description (schema coverage 0%). The description adds no meaning beyond the parameter name; it does not explain what valid entity IDs look like, what types of entities are accepted, or how to specify them. Thus, it provides no additional semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Toggle') and the resource ('entity') with specific outcome ('between on and off'). It effectively distinguishes from siblings like entities_turn_on and entities_turn_off, which target specific states rather than flipping. The verb+resource+scope is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as entities_turn_on, entities_turn_off, or entities_enable_entity. It does not explain that toggle is suitable when the current state is unknown or when a simple state flip is desired. The agent must infer usage context from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. The description states it 'gets' definitions, implying a read-only operation, but fails to mention any potential side effects, authentication requirements, rate limits, or whether the device must be online. For a simple getter, this is a minimal but insufficient disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of a single sentence that front-loads the key information. Every word serves a purpose. However, it could be slightly improved by including a brief usage note without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is an output schema (mentioned in context), the description does not need to detail return values. However, it lacks important context such as prerequisites (e.g., the device must be configured with LVGL), error scenarios, or whether the device must be online. For a tool with only one parameter, this is minimally adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'device_name', has no description in the schema (0% coverage), and the tool description adds no additional meaning. It does not explain what constitutes a valid device name (e.g., friendly name vs. internal ID) or any expected format. The description fails to compensate for the schema's lack of documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get LVGL theme and style definitions from an ESPHome device config.' It uses a specific verb ('get'), identifies the resource ('LVGL theme and style definitions'), and specifies the source ('ESPHome device config'). This clearly distinguishes it from sibling tools like 'esphome_lvgl_get_pages' and 'esphome_lvgl_get_page_widgets'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus related sibling tools such as 'esphome_lvgl_get_page_widgets' or 'esphome_lvgl_get_pages'. It does not mention any prerequisites, typical use cases, or contexts where this tool is appropriate. This lack of usage guidance reduces its helpfulness for an AI agent deciding between tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description only states the action 'cancel', omitting details like what happens if the timer is already expired or not running. No annotations to supplement behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, clear sentence with no superfluous words. Efficiently communicates the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks detail on expected behavior, error cases, or parameter meaning. Minimalist description leaves agents underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention the required 'entity_id' parameter at all. With 0% schema description coverage, the agent gets no help understanding what value to provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the specific verb 'cancel' and resource 'a running timer', distinguishing it from siblings like helpers_start_timer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use versus alternatives, prerequisites (timer must be running), or when not to use. Implied by the verb but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
While the description correctly implies a write/destructive operation, it does not disclose any side effects, permission requirements, or limitations. With no annotations, more detail is expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence), but it is so brief that it omits useful context. It is front-loaded but arguably too minimal for effective agent use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool like resetting a counter, the description covers the core functionality, but it does not mention return values or confirm that the operation succeeded. The presence of an output schema partially mitigates this.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'entity_id' has no schema description and the tool description does not explain its meaning or how to obtain valid values. The description adds no value beyond the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (reset) and resource (counter helper) and the resulting value (to 0). The description is specific and distinguishes from sibling tools like helpers_increment_counter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides no guidance on when to use this tool versus alternatives such as helpers_increment_counter or other helpers_set_* tools. Lacks any context about prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states the basic function but omits details like authentication needs, rate limits, behavior for missing entities, or data format. The existence of an output schema is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence of 13 words conveys the core purpose without redundancy. It is front-loaded and every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity and an output schema, the description lacks important context such as what the history includes (e.g., state changes, timestamps), any limits on the 'hours' parameter, and whether the output is paginated. The agent would need to infer behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the schema. It does not explain that 'entity_id' is the ID of the entity whose history is retrieved, nor does it clarify the 'hours' default or its interpretation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'state history for an entity', and the time scope 'over the last N hours'. It uniquely identifies the tool among siblings like 'history_get_error_log' and 'history_get_logbook'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description does not mention prerequisites, when not to use it, or scenarios where other history tools would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states it updates properties, but fails to mention idempotency, side effects, required states, or response behavior. Output schema exists but is not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words, achieving high conciseness. However, it sacrifices informativeness, so while structurally efficient, it is under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no annotations, and an output schema, the description is too brief. It does not explain what a config entry is, when updates take effect, or error cases, leaving significant gaps for safe usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds minimal meaning: it groups 'polling preferences' but does not explain individual parameters like 'pref_disable_new_entities'. The mapping is incomplete and lacks detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'config entry', and specifies the properties 'title' and 'polling preferences'. This differentiates it from sibling tools like disable, enable, or remove integration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or exclusions. The description is minimal and does not help the agent decide context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It does not disclose behavioral traits such as idempotency, side effects, or error conditions. Mention of the WS endpoint provides minimal context but is insufficient for transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but overly terse. It lacks critical information that could be added without bloat. It is front-loaded with the action but not structured to be helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists (so return values need not be described), the description omits parameter syntax, usage context, and behavioral details. For a simple tool with three parameters, this level of completeness is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only implies the role of 'scope' and 'name' via the phrase 'in the given scope', but provides no details on icon, acceptable values, or format. This adds little beyond the schema field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a category in the given scope', which is a specific verb-resource combination. It distinguishes from sibling tools like labels_create_label (creates a label) and labels_update_category (updates a category).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., labels_create_label, labels_update_category). No mention of prerequisites, scope restrictions, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only mentions 'fuzzy search' without revealing side effects (likely read-only), required permissions, or algorithmic details. The behavior is not clearly defined beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but overly brief. It could incorporate additional useful context without becoming verbose, such as mentioning output or usage hints. Front-loaded structure is fine but too sparse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, output schema exists) and lack of annotations, the description is incomplete. It does not explain what 'areas' are, the return format, or any behavioral constraints, leaving significant gaps for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description should explain parameters. It implies 'query' is a name for fuzzy matching and 'limit' controls result count, but it does not clarify the fuzzy algorithm, input format, or edge cases. Minimal value added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('search'), the resource ('areas'), and the method ('fuzzy by name'). It distinguishes from sibling search tools like search_search_devices and search_search_entities by specifying the resource, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. With many sibling search tools (e.g., search_deep_search, search_find_related), the description should note scenarios or prerequisites, but it does not.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only says 'Toggle play/pause' but does not disclose behavioral details such as state changes, potential side effects, or response behavior. Without annotations, the description should provide more insight but fails to do so.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) and front-loaded, but it is under-specified. It is concise but missing essential details, earning a middle score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single required parameter and absence of annotations or output schema explanation, the description is incomplete. It does not explain what kind of media player, where to find entity_id, or what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter 'entity_id' with no description in the schema (0% coverage). The description does not mention this parameter, leaving the user to infer that it identifies the media player. This adds no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Toggle play/pause') and the resource ('on a media player'). It is specific and distinguishes from sibling tools like 'services_media_seek' or 'services_set_volume'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, such as the entity must be a media player, or when not to use it (e.g., if seeking or volume adjustment is needed).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fails to disclose behavioral traits such as whether the light must already be on, whether multiple entities can be set, or side effects. Only states the action without further context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 9 words, efficient and without fluff. However, it could be structured to front-load key constraints or provide a quick summary of parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters (1 required), no annotations, and no output schema details provided, the description is too sparse. It does not cover usage context, behavioral expectations, or parameter completeness, relying excessively on the schema's implicit structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'light color, color temperature and/or brightness' but does not explain valid ranges (e.g., RGB integers, Kelvin, brightness percentage) or that entity_id must be a light entity. Adds minimal value over parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Set') and the resources affected (light color, color temperature, brightness). It distinguishes from sibling tools like services_set_climate_mode or services_set_cover_position by specializing in light attributes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool instead of alternatives such as entities_turn_on or services_call_service. Prerequisites (e.g., light should be on) are not mentioned, and there is no comparison to sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but only implies a read operation. It does not mention permissions, rate limits, data volume, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with 12 words, front-loading the key action. It is appropriately sized but could include more specifics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is an output schema, so return values need not be described. However, the description is minimal and does not clarify what 'aggregated statistics' entails or the exact semantics of the time range, leaving gaps for a tool with 5 parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 5 parameters described), so baseline is 3. The description adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'aggregated statistics for one or more entities over a time range.', but it does not differentiate from siblings like 'statistics_get_energy_statistics' or 'statistics_get_statistics_metadata'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must convey behavioral traits. It states 'Delete' implying irreversibility, but does not mention permissions, side effects, or error handling. With zero annotation coverage, the description carries full burden but fails to add significant value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at one sentence, front-loaded with key action and resource. However, it sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete tool with one parameter, no annotation coverage, and an output schema (content unknown), the description is insufficient. It omits what the tool does beyond the name, what output to expect, and how to handle failures.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description should explain parameter 'slug'. It mentions 'by slug' but does not define what a slug is or how to determine valid values. The schema offers no additional description, leaving the agent underinformed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Delete' and resource 'backup' with qualifier 'by slug', clearly distinguishing it from related tools like create, list, and restore.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., supervisor_restore_backup), or any prerequisites or cautionary notes. The agent is left without context for decision making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states the core action without mentioning side effects, permission requirements, or whether the add-on needs to be restarted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 11 words, front-loaded with the core purpose. It is concise and avoids redundancy, though it could be slightly expanded without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two required parameters (one an arbitrary object) and no annotations, the description is incomplete. It fails to explain what ‘slug’ is, valid option keys, or the outcome of the operation, even though an output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% (no parameter descriptions). The description adds a format hint for 'options' ('sent as {"options": options}') but ignores the required 'slug' parameter entirely, leaving it undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set configuration options for an add-on' uses a specific verb ('Set') and resource ('configuration options for an add-on'), clearly distinguishing it from siblings like 'get', 'install', or 'start' add-on tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., supervisor_update_addon), no prerequisites (e.g., if the add-on must be stopped), and no exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden but only states the action. It does not disclose behavioral traits like permission requirements, whether it restarts the add-on, or what happens if the update fails. This is insufficient for an AI agent to understand side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 8 words, making it very concise. It front-loads the action and resource, but the extreme brevity may sacrifice clarity. Still, it is appropriately sized for such a simple operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain the update process, return values, or potential side effects (e.g., add-on restart). Given the lack of annotations and parameter details, the description is incomplete for an agent to safely invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the single parameter 'slug'. The description does not explain what 'slug' refers to (e.g., add-on identifier). Without compensating parameter description, the agent lacks semantic understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'add-on', specifying it updates to the latest available version. This distinguishes it from sibling tools like install_addon or restart_addon.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when an add-on is already up-to-date or if it needs to be installed first. The description lacks context about prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states it sets the theme but doesn't mention persistence, scope (user vs global), side effects, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with two sentences, no filler. Slightly too brief to be optimal; could add a bit more context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description needn't detail returns, but it lacks completeness about the effect of the operation, error conditions, or relation to other theme tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should explain all parameters. It explains 'mode' values 'light' or 'dark' but does not describe the 'name' parameter (what 'default' means, which theme it refers to).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Set' and clearly identifies the resource as 'the active frontend theme'. It distinguishes from sibling tools like themes_create_theme or themes_delete_theme by indicating this is for setting the active theme.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like themes_list_themes or themes_create_theme. No prerequisites or context on which theme names are valid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It only states 'Get' and 'WebSocket', omitting details like read-only nature, authorization needs, or possible side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one short sentence), but it lacks necessary context. While not verbose, its brevity sacrifices completeness for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no annotations, the description should provide more context about what 'HA config' entails and how this tool differs from other config-retrieving siblings. It is insufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100% (vacuously). Per guidelines, baseline is 4 for no parameters. The description adds no parameter info but does not need to.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get HA config' and the method 'via WebSocket'. While it does not distinguish from sibling tools like history_get_ha_config, it is specific enough about the resource and mechanism.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as history_get_ha_config or system_check_config. The description lacks context on preferred scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It states it 'waits' and 'returns trigger context', but does not mention potential blocking, timeout handling, or what happens if the trigger never fires. The timeout parameter is in schema but not explained behaviorally.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences plus an example, which is efficient and front-loaded. It wastes no words, though the example could be integrated more tightly. Overall, it is appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, return values are not needed, but the description still misses usage guidance and behavioral details. With 0% schema coverage, it should explain parameter semantics more thoroughly. It is not complete enough for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the description adds no parameter details beyond the schema. While the example shows a sample trigger object, it does not explain the structure or required fields. The timeout parameter is not mentioned at all. The description fails to compensate for low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Wait for a HA trigger to fire' and specifies the action of waiting and returning trigger context. It provides an example which clarifies the resource. Among siblings like ws_listen_events and ws_listen_state_changes, this tool is uniquely defined by its trigger-based waiting semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives. For example, it does not mention that it is a long-polling operation that may block, or compare to streaming state changes via ws_listen_state_changes. The description lacks context on appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description fails to disclose behavioral traits such as read-only nature, required permissions, or idempotency. It only states what the tool returns, not its behavior beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of two short sentences that front-load the main purpose. However, the brevity sacrifices clarity on parameter semantics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although the tool is simple with one parameter and an output schema exists, the description fails to explain the input parameter 'name' and its relationship to the sibling tool, leaving gaps in understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not add meaning to the sole parameter 'name'; it only says 'Full design pattern entry' without explaining what the parameter represents or its possible values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a 'Full design pattern entry' and distinguishes it from the sibling tool 'list_design_patterns' by referencing 'same fields', indicating a single item retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied via the reference to `list_design_patterns`, suggesting to use this for a single full entry and the sibling for listing, but no explicit when-to-use or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It indicates a read operation and mentions returns widget counts and background colors, but fails to mention if authentication is needed, performance characteristics, or potential failures.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the core purpose and key outputs without unnecessary words. It is front-loaded with the verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and an output schema present, the description covers the return information (pages, widget counts, background colors). However, it lacks parameter documentation and usage context, making it incomplete for full autonomous use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema documentation coverage is 0%, and the description does not explain the device_name parameter. It only implies the device context but does not clarify format, constraints, or examples, leaving the agent without sufficient guidance for parameter input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'LVGL pages', specifies the scope (configured on an ESPHome device) and additional details (widget counts, background colors). It distinguishes from sibling tools like esphome_lvgl_get_page_widgets which retrieves widgets for a specific page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives, no prerequisites, no exclusions. For example, it doesn't clarify whether get_pages should be used first to get available pages before using get_page_widgets.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden but only states the action. It does not disclose side effects (e.g., discarding changes), whether user interaction is required, or any permissions needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (one sentence) but under-specified for a tool with no annotations. It sacrifices necessary detail for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no annotations and an output schema exists, the description fails to explain return values or behavior. It lacks context about what 'aborting' entails (e.g., validation, user prompts).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the schema. However, the single 'flow_id' parameter is self-explanatory by name, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Abort') and the resource ('options flow'), distinguishing it from sibling tools like 'integrations_abort_config_flow' and 'integrations_submit_options_flow_step'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to abort vs. submit. Prerequisites or exclusions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description partially compensates by noting the filename path constraint and directory allow-listing requirement. However, it does not disclose side effects (e.g., file overwrite behavior), permissions needed, or output format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two focused sentences. It is front-loaded with the core purpose, then adds key detail about the filename. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool, the description covers the filename constraint but omits the return value (though output schema exists), entity_id expected values, and typical usage patterns. It is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for the 'filename' parameter (path inside HA config dir, directory must exist and be allow-listed) but offers nothing for 'entity_id'. Since schema description coverage is 0%, the description should document both parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Take a still snapshot from a camera entity') and defines the primary resource. However, it does not distinguish this from the sibling tool 'services_camera_record', which is a closely related alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., 'services_camera_record'). The description mentions directory requirements but lacks explicit usage context or preconditions beyond those.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description discloses the core action (creates a saved card) and the slot override behavior, but does not mention side effects, permissions, or error states. Acceptable but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. First sentence states the core action and result; second sentence adds critical usage context for the output. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main intent and output usage but omits parameter details for optional fields and does not mention error handling or prerequisites. For a tool with 6 params and an output schema, it is partially complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain all parameters. It only describes 'template', 'name', and 'slot' (implicitly), leaving 'description', 'tags', and 'categories' unexplained. Minimal compensation for schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: pick a template, name it, get back a card_id. It distinguishes from siblings like card_builder_create_card by specifying it's a template card creation, but does not explicitly differentiate from other card_builder tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like card_builder_create_card or card_builder_build_from_recipe. The description implies a one-shot creation but does not exclude other use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that content_base64 must be pre-encoded and specifies the return format. However, it does not mention potential side effects (e.g., overwriting), permissions, size limits, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the purpose, the second adds critical encoding and return info. It is front-loaded, concise, and every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 params, no annotations, but has output schema), the description covers the encoding requirement and return format but omits details about the path parameter, error handling, and any constraints on filename or content size.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains content_base64 (must be base64-encoded) but does not describe filename or path. The path parameter is left undefined, and no constraints (e.g., allowed extensions) are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Upload a file') and the target resource ('Card Builder media directory'). It mentions the encoding requirement and return format, which helps distinguish it from sibling tools like card_builder_upload_image_from_url or card_builder_upload_media_from_path, but does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, if the user has a URL or file path, they should use other tools. The description lacks any when-to-use or when-not-to-use information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It only says 'Add a new view,' indicating a create operation, but lacks details on permissions, side effects (e.g., whether it modifies a dashboard config), or idempotency. The tool likely writes data, but no warnings or clarifications are given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short, with one sentence for purpose and one for an example. Every word is useful, but it could be more structured, e.g., separating parameter details or adding a hint about the output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool having two parameters (one nested) and no annotations, the description is minimal. It lacks details on how to construct view_config beyond the example, what the tool returns (though output schema exists), error cases, or integration with other dashboard tools. For a mutation tool modifying dashboard layout, more completeness is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has two parameters with 0% description coverage. The description partially compensates by providing an example of the 'view_config' parameter, showing it expects title, icon, and cards. However, it does not explain the 'url_path' parameter (the dashboard identifier) or the full structure of view_config (e.g., valid fields, card format).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a new view to a dashboard,' using a specific verb and resource. It distinguishes from sibling tool 'dashboards_add_card_to_view' which adds cards to an existing view, so the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention prerequisites, such as requiring an existing dashboard, nor does it specify when to prefer this over other dashboard tools. The example provides some usage context but no explicit conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It only states 'validate', but does not disclose what checks are performed, whether it is read-only, or what happens on failure. Minimal behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence (11 words) efficiently communicates the tool's purpose. No wasted words, includes WS reference. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain what validation returns or what constitutes a valid/invalid configuration. The tool's purpose is clear, but the agent lacks understanding of expected results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero parameters, so no parameter description needed. Schema coverage is 100%. The description adds no parameter info, but that is acceptable given the absence of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (validate) and resource (Energy Dashboard configuration), and references the WebSocket command. It distinguishes from sibling tools like 'energy_get_energy_prefs' by the validation action, but does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like energy_get_energy_prefs or energy_save_energy_prefs. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the action without revealing what happens if the entity is already enabled, authorization requirements, or side effects. For a simple tool, this is still minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with one sentence and no waste. For a simple tool, this is appropriate, though it may be under-specified for more complex scenarios.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (enable an entity) and the presence of an output schema, the description is somewhat complete. However, it lacks context about edge cases (e.g., re-enabling an already enabled entity) or what the output signifies. It meets minimal viability but has gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for its single parameter. The description does not add any meaning beyond the parameter name 'entity_id', only implying through context that it identifies the entity to re-enable. It insufficiently compensates for the schema's lack of parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: re-enable a disabled entity. It uses a specific verb and resource, and implicitly distinguishes from the sibling 'entities_disable_entity'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no context for when it should not be used. It is a single sentence without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It implies a read-only operation ('Get') but does not specify whether the tool has side effects, requires special permissions, or what the response format includes. The description adds minimal value beyond the action itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded. However, given the lack of detail elsewhere, it could be slightly expanded without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and an output schema, the description adequately states the core purpose but omits details about return values or behavior. The presence of siblings and the absence of annotations lower completeness, but the tool is simple enough that the description is minimally viable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'entity_id' with no description (0% schema coverage). The description does not clarify expected format or domain for entity_id, such as whether it should be an entity registry ID or a friendly name. The parameter name is self-explanatory, but the description fails to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get' and clearly identifies the resource 'exposure flags' for a single entity, distinguishing it from siblings like 'entities_set_entity_exposure' and 'entities_list_exposed_entities'. It also specifies the scope 'across all voice assistants', which adds useful context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives such as 'entities_list_exposed_entities' (for all entities) or 'entities_set_entity_exposure' (for modifying exposure). The description does not mention prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavior. It notes that lambda callbacks are shown as '<lambda>' and directs to esphome_get_config for details. However, it doesn't state whether the operation is safe (non-destructive) or require authentication, though the name implies a read.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, return summary, and a notable behavioral detail. No redundancy, information density is high.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 simple parameters, no annotations, but an output schema exists, the description covers the core functionality and a key behavioral nuance. However, it omits parameter descriptions and usage context, leaving gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description does not explain the parameters 'device_name' or 'page_id'. It mentions 'a specific page of an ESPHome device' but provides no format or context for the parameter values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves all widgets on a specific page, listing the type of information returned (types, IDs, positions, key properties). It distinguishes from sibling tools like add/delete widget, though not explicitly from other get tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives, nor prerequisites like device connectivity or page existence. The description implies a read operation but lacks situational context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It only states 'Abort a config flow in progress' without explaining side effects, whether resources are cleaned up, or if the flow can be aborted at any step. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence of six words, perfectly concise and front-loaded. Every word is necessary and serves the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 1 parameter and no annotations, the description is too sparse. It omits context about when a config flow can be aborted, confirmation of success, and any implications. While output schema exists, behavioral completeness is lacking.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for the flow_id parameter. The description does not explain what flow_id represents or how to obtain it, leaving the agent to infer from the parameter name alone. The description adds no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Abort a config flow in progress,' using a specific verb and resource. It distinguishes this tool from sibling tools like start_config_flow, submit_config_flow_step, etc., by its action of aborting an existing flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description doesn't indicate when to use this tool versus alternatives (e.g., when to abort vs. remove an integration), nor does it mention prerequisites or post-conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states that the entry is not removed, but does not explain what disabling entails (e.g., effects on services, reversibility, permissions). This is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no waste. It could be expanded with critical details without losing conciseness, but it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter), the description is minimal but lacks details about the output schema or return value. It is adequately complete for a basic tool but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for the parameter 'entry_id', and the description does not mention or explain it. The description adds no meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Disable' and the resource 'integration config entry', and explicitly distinguishes from removal by adding 'without removing it'. This distinguishes it from siblings like integrations_remove_integration and integrations_enable_integration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like enable or remove. There is no mention of prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries full burden. It indicates a destructive action but does not disclose details like permanence, data loss, or system impact. The confirm flag behavior is ambiguous (what happens if false?).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler. Purpose stated first, usage hint second. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema and sibling tools, the description lacks details on return values, error handling, and context for choosing this over related integration tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must explain parameters. It only mentions confirm=True, ignoring entry_id entirely. The required parameter is left unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Remove (uninstall) an integration config entry.' It uses a specific verb and resource, distinguishing from sibling tools like disable, enable, or update integration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Only guidance is 'Set confirm=True to proceed.' No information on when to use this tool over alternatives (e.g., disable, update), prerequisites, or potential side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full burden. It mentions listing via a WS command but lacks disclosure of read-only nature, pagination, error handling, or response structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, efficient sentence conveys the tool's purpose and endpoint reference without waste, but could be slightly more structured with separate purpose and usage notes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description is minimally adequate but does not elaborate on categories, scope values, or what the output contains, leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the purpose of the 'scope' parameter with an example ('automation'), adding meaning beyond the schema's default string type, though schema coverage is 0%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists categories for a scope, with an example ('automation') and references the specific WS endpoint, distinguishing it from sibling label operations like create/delete or listing labels.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as labels_list_labels or other list tools. No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It only states it lists services but does not disclose what is returned (service names, details), authentication needs, or error behavior. This is insufficient for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence, front-loading the key action. It is efficient, though it could be slightly expanded for completeness without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not mention what the tool returns (e.g., list of service names). For a list tool among many siblings, more context about the result would improve completeness. Score 3 as it is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only adds 'optionally filtered by domain' without explaining what domain means, acceptable values, or format. For a single parameter, this adds minimal semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'all available services', and the optional filter by domain. It is specific and distinguishes from sibling tools like services_call_service.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention when not to use it or provide any usage context relative to other services_* tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavioral traits. It only says 'reload' without explaining effects like potential downtime, impact on running processes, or safety. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise single sentence, front-loaded with action. However, it sacrifices completeness for brevity, missing important behavioral details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no params, has output schema), the description is adequate but lacks any cautionary notes or explanation of the scope of 'core configuration'. Could mention if it restarts any services.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds value by listing what gets reloaded, which is helpful context even though there are no parameters to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it reloads core configuration and lists examples like automations, scripts, scenes, groups. It is distinct from sibling tools that target specific subsystems, but does not explicitly differentiate itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like automations_reload_automations. Implied usage from name and description, but no explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. 'Reload' implies re-reading configuration, but it does not disclose potential side effects (e.g., overwriting existing zones, impact on active automations, error handling). Minimal information beyond the basic operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—single sentence with no extraneous words. Every word ('Reload', 'zones', 'from YAML', 'via service zone.reload') carries essential information. Appropriately sized for a simple action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (which reduces the need to explain return values), the description lacks critical context: what triggers a reload, whether it requires a specific file, behavior on error, and relationship to other zone operations. For a system action, more completeness is warranted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description does not need to add parameter detail. However, it adds meaning beyond the empty schema by specifying the source (YAML) and service. This is a reasonable baseline for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (reload) and resource (zones from YAML). It specifies the underlying service (zone.reload). While it doesn't explicitly differentiate from other reload tools, the tool name and context make the purpose sufficiently clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like zones_create_zone, zones_update_zone, or other reload tools (e.g., automations_reload_automations). The description does not provide context for when a reload is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states 'delete', implying destruction. It does not disclose consequences (e.g., impact on associated entities), reversibility, or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence (6 words) with no unnecessary information. It is front-loaded and directly communicates the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an output schema, the description is adequate but lacks usage context and behavioral details, leaving the agent without critical decision support.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and while the description mentions 'area_id', it adds minimal meaning beyond the schema's type definition. No format, source, or examples are provided for the single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete an area by its area_id' clearly states the action (delete) and the resource (area), with the specific method (by ID). It distinguishes from sibling tools like areas_create_area or areas_control_area.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool, prerequisites, or alternatives. The description does not help the agent decide between this tool and others like areas_control_area.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions the destination path but does not disclose side effects (e.g., overwriting existing blueprints), error handling, or authorization requirements. This leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, efficient sentence that front-loads the action and destination. No redundant words, making it highly scannable for an AI agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks context on expected return values or error conditions. For a simple import tool, it is minimally adequate but could be more complete by clarifying side effects or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description only mentions 'URL' generically and does not explain the 'domain' parameter (default 'automation') or any additional context. The description adds no semantic meaning beyond the schema's structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Import' and the resource 'blueprint from a URL' with a specific destination path. It distinguishes from sibling tools like blueprints_list_blueprints or blueprints_delete_blueprint by focusing on importing from external sources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for importing blueprints from URLs but does not provide explicit guidance on when to use this versus sibling tools like blueprints_substitute_blueprint. No 'when not to use' or alternative suggestions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes the override behavior (later snippets override earlier ones at property level) and the shape of extra parameter. However, no annotations are provided, and the description doesn't mention side effects, permissions, or preconditions. The read-only nature is assumed but not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise with a clear example. It front-loads the purpose and structure. Slightly longer due to example, but still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description omits details on target and container parameters, and does not fully compensate for low schema coverage. While it has an output schema (not shown), the example output gives some context, but missing parameter information reduces completeness for a 4-param tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains snippet_names and extra, but does not mention target or container parameters (both have defaults). The description doesn't specify that snippet_names expects a list of strings or extra's full schema beyond a partial example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it composes a styles object from snippets and overrides, with a specific verb ('Compose') and resource ('styles object'). It distinguishes itself from sibling tools like card_builder_get_style_snippet by focusing on building a combined style.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives (e.g., card_builder_build_from_recipe). The description implies it's for combining snippets, but doesn't mention when not to use it or other options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must fully disclose behavior. It only states 'Delete' without mentioning permanence, cascading effects, or permissions, leaving major gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient phrase with no unnecessary words. It wastes no space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an output schema, the description is adequate but lacks context on error conditions or prerequisites like the property existing. It meets minimum viability.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain the parameter. It does not; 'custom_property_id' is self-explanatory but no additional context (e.g., how to obtain it) is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a CSS custom property' uses a specific verb (delete) and resource (CSS custom property), clearly distinguishing it from sibling tools like create or list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. The agent is left to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, permission requirements, or side effects. The name implies retrieval, but explicit safe operation is not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the core purpose ('Full schema for one block type') and efficiently lists key attributes without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, output schema exists), the description adequately conveys the output content but lacks usage context, prerequisites, or instruction on how to proceed after obtaining the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only implies the parameter 'block_type' selects a block type. It does not specify valid values, format, or how to obtain them, leaving the agent to infer from context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Full schema for one block type' with a detailed list of contents (props, types, defaults, enum values, notes, supported domains, style targets), clearly distinguishing it from siblings that list block types or retrieve other card configurations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like card_builder_list_block_types. The description only states what it returns, not the context or prerequisites for invoking it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses the mutation (save) and partial update behavior, which adds context. However, it does not mention authorization, rate limits, or side effects, leaving gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise—one sentence with a parenthetical—and front-loads the purpose and partial update behavior. It is efficient, but could be slightly expanded to include parameter guidance without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given four parameters with no schema descriptions and an output schema, the description is incomplete. It does not explain what each parameter does or describe the return value, leaving critical gaps for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain parameters. It only mentions 'fields you pass' without defining any of the four parameters (e.g., energy_sources, currency). This fails to add meaningful semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it saves Energy Dashboard preferences and specifies the WebSocket endpoint. It also distinguishes from siblings like energy_get_energy_prefs and energy_validate_energy_prefs by implying a write operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating preferences by noting 'only the fields you pass are sent,' suggesting partial updates. However, it does not explicitly state when to use this vs. other energy tools (e.g., get, validate) or provide exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the action without disclosing side effects, authorization needs, what 'stale' means, or whether the operation is reversible. For a deletion tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) and front-loaded with the action. However, it lacks structure (e.g., breaking into purpose, usage, behavior) and could be more efficient by including key details without added length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, return values are covered. However, with no annotations and a single parameter, the description is incomplete. It does not specify prerequisites, when to run, or how 'stale' is determined, leaving gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions the tool is 'for an ESPHome device' but does not explicitly map the 'name' parameter to a device name or provide format details. The description adds minimal meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove'), the resource ('stale MQTT discovery entries'), and the target ('for an ESPHome device'). It also specifies the mode ('MQTT mode only'), distinguishing it from other ESPHome tools like compile, upload, or validate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for MQTT mode only but does not explicitly state when not to use it or name alternatives. No guidance on prerequisites or context beyond the mode qualification.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only implies a non-destructive read. It does not disclose error behavior, file size limits, encoding, or what happens if the file does not exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence plus an example. No unnecessary words or repetition. Efficiently conveys the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 1-parameter tool with an output schema, the description is mostly adequate but lacks usage context relative to numerous sibling tools. It does not guide the agent on when to select this over similar read operations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to 'relative_path' by clarifying it is relative to the HA config directory and providing examples. Schema coverage is 0%, so this compensation is valuable, though it could mention path syntax or allowed values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads a config file relative to the HA config directory, with concrete examples. It distinguishes from write/delete/list tools but does not explicitly differentiate from other read tools like automations_get_automation_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., files_list_config_files, automations_get_automation_config). The description only states what it does, not when it should be chosen.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description must carry full burden. It discloses the mutation (re-enable), but omits prerequisites, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single 8-word sentence, front-loaded with key action and object, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity and presence of output schema, the description minimally covers what the tool does but lacks details on effect and appropriate context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description adds no meaning beyond the parameter name 'entry_id'. With no compensation, the parameter is effectively undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Re-enable a previously disabled integration config entry' uses a specific verb ('Re-enable') and resource ('integration config entry'), clearly distinguishing it from sibling 'integrations_disable_integration'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use after an integration has been disabled, but provides no explicit guidance on when or when not to use it, nor alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It fails to disclose that starting a flow typically returns a flow_id needed for subsequent submit calls, or that the flow may involve multiple steps. The agent is left unaware of the flow lifecycle.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that is concise and front-loaded. It could be slightly expanded to add behavioral context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a multi-step options flow and the presence of sibling tools for submit and abort, the description is incomplete. It does not mention the return value (likely a flow object) or the need to continue with submit_options_flow_step. The output schema exists but is not described, leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain the two parameters ('entry_id' and 'show_advanced_options'). It only mentions 'existing config entry' but does not clarify that 'entry_id' is the ID of that entry. No meaning is added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it starts the options flow for an existing config entry, explicitly adding 'edit settings' in parentheses. This distinguishes it from the sibling 'integrations_start_config_flow' which is for new integrations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for editing existing configuration entries, which differentiates it from starting a new config flow. However, it does not explicitly state when NOT to use it or mention alternatives like 'abort_options_flow' or submit steps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions return types but does not describe side effects, idempotency, or state changes, which is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the action, and contains no unnecessary words. It is concise yet informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values are covered. However, the description lacks context about parameter usage, error handling, and prerequisite flow state, making it only minimally adequate given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), and the tool description does not explain what flow_id or user_input represent or how to structure user_input, leaving the agent without guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool submits user input for a config flow step and lists possible outcomes. It distinguishes from siblings like start_config_flow and get_config_flow by specifying 'current step' and 'config flow'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used after starting a config flow and receiving a step form, but it does not explicitly state when to use it over alternatives or provide prerequisites/instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear the full transparency burden. The description accurately implies a read-only operation but adds no further behavioral context (e.g., permissions, side effects, or error states). It is adequate for a simple list operation but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence containing all essential information. It is efficiently front-loaded with the verb and resource, with no redundant words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Considering the simplicity of the tool (one parameter, read-only output), the description covers the basic purpose but lacks parameter semantics and usage context. The presence of an output schema helps, but the missing parameter guidance reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% parameter description coverage, and the tool description does not explain the parameter 'label_id' (e.g., format, example, or how to obtain it). The agent is left guessing the correct value, which is a critical deficiency for correct tool invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action (list), the target resource (devices), and the condition (having the given label). It distinguishes this tool from siblings like labels_list_labels (lists labels) and labels_list_entities_with_label (lists entities, not devices), leaving no ambiguity about what it does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., label must exist), nor does it suggest when not to use it (e.g., for entities instead of devices). Given many similar sibling tools, explicit usage guidance is lacking.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'Dismiss' without explaining side effects, permissions needed, or error handling (e.g., invalid notification_id). The description is too minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It efficiently communicates the core action and required parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool, the description is minimally sufficient but lacks context about related tools (create, dismiss_all) and behavioral details. Could be improved with minimal additions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate. It only mentions `notification_id` in backticks without adding any meaning like format, example, or source beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Dismiss) and the resource (persistent notification), and explicitly mentions the key parameter `notification_id`. It distinguishes itself from siblings like services_notify_persistent_create and services_notify_persistent_dismiss_all.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives. There is no mention of prerequisites (e.g., notification must exist) or that a dismiss_all option exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It states a read-like operation but offers no details on performance, result size, or other behaviors. The presence of an output schema helps, but the description itself adds minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 10 words, front-loads the verb 'Get' and the resource 'all active config entries'. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter retrieval with output schema, the description adequately explains the tool's purpose. However, it could be more complete by distinguishing from similar sibling tools like system_list_integrations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema description coverage is 100%. Baseline is 3 as per guidelines; the description correctly omits parameter details since none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves all active config entries for integrations, specifying state and domain. However, it doesn't explicitly differentiate from the sibling tool system_list_integrations, which might overlap in purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like system_list_integrations. The description provides no context for choosing this over other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full behavioral burden. It only mentions the action and a due_date format hint, lacking details on side effects, error cases, or entity_id validation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (2 sentences) with no fluff, but it could be better structured by front-loading the core purpose and then explaining parameters. It is efficient but not optimally organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity and the presence of an output schema, the description is incomplete: it fails to clarify what 'entity_id' (likely a todo list identifier) and 'item' (the todo text) mean, leaving ambiguity for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain all parameters. It only clarifies 'due_date' format, leaving 'entity_id', 'item', and 'description' completely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Add a new item to a todo list', which is a specific verb+resource pair. It clearly distinguishes from sibling tools like todo_remove_item and todo_list_items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for adding items but provides no explicit guidance on when to use this tool versus alternatives, nor any when-not-to conditions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the core function without disclosing side effects (likely read-only), required permissions, error handling, or expected behavior if zone does not exist or is empty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant information. It is front-loaded with the verb and resource, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (1 parameter, output schema exists), the description is somewhat complete in stating the basic function. However, it lacks behavioral context such as read-only nature, potential errors, or expected performance. Adequate but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one parameter (zone_entity_id) with no description. The description does not explain the parameter format or naming conventions (e.g., 'zone.home'). The schema description coverage is 0%, so the description should compensate but fails to add meaningful detail about the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('person.* entities'), and clarifies the matching mechanism ('matched on friendly_name → state'). It clearly distinguishes from sibling tools like zones_list_zones (which lists zones) and zones_get_user_location (gets location of a specific user).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as zones_get_user_location for a specific person or zones_list_zones to get zone info. There is no mention of prerequisites, when not to use it, or comparison with similar tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description tells what the tool returns (steps, timing, condition results) but omits prerequisites, error conditions, or permissions. It is adequate but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single concise sentence that conveys the core purpose. Could be slightly expanded to include usage context, but not overly long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read tool with output schema, the description covers what is returned but omits relationship to listing traces and dependencies. Adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the parameter names. 'automation_id' and 'run_id' are not explained in terms of format or source.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a full automation trace for a specific run, listing components (steps, timing, condition results). It distinguishes from siblings like automations_list_automation_traces (listing) and automations_get_script_trace (scripts).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as needing to first list traces to get a run_id, or differences from automations_get_script_trace. The description lacks context for proper selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the auto-reload behavior and provides the entities format with brightness range, but fails to mention permissions, what happens on overwrite, return values, or potential side effects like impact on active scenes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: one sentence for purpose and action, then a clear block for entities format. There is no fluff, though it could be more structured with sections.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero annotations and a complex object parameter, the description omits important context: what happens when scene_id is new vs existing, any prerequisites, error conditions, and the return value (though output schema exists). The tool's behavior around overwrite and reload is only partially explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must clarify parameters. It provides a detailed format and example for the entities object with brightness mapping, but does not add semantics for scene_id or name (e.g., length limits, uniqueness rules).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create or overwrite a scene config', using a specific verb and resource. It also mentions auto-reloading scenes, which distinguishes it from related tools like automations_activate_scene or automations_get_scene_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lacks any guidance on when to use this tool vs alternatives (e.g., automations_activate_scene for activation, automations_get_scene_config for reading). It does not specify prerequisites, nor when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states it creates an event and returns a dict, but does not disclose side effects (e.g., calendar integration, permissions required, or potential errors). Minimal behavioral disclosure for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the purpose and a critical detail about datetime format. No redundant information; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (create event with multiple fields) and the presence of an output schema, the description is too minimal. It omits explanation of entity_id, return value details, and any behavioral constraints, making it incomplete for an agent to use correctly without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains start and end format (ISO 8601) but does not describe entity_id, summary, description, or location. Only 2 of 6 parameters gain meaning, leaving significant gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a calendar event via a specific service. It specifies the datetime format for start/end and distinguishes from sibling tools like calendar_delete_event and calendar_list_events by being a creation operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through the verb 'create' and mentions the datetime format, but does not explicitly state when to use this tool vs. alternatives or provide any prerequisites or exclusions. Usage is clear from context but not explicitly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as whether the deletion is permanent, reversible, or what happens if the path does not exist. For a destructive operation, more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no redundant information. It is front-loaded and efficient, earning its place without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, output schema exists), the description covers the basic semantics but lacks contextual completeness: no guidance on error handling, prerequisites, or relation to sibling tools. The gaps in behavioral transparency and usage guidelines reduce completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter ('path') is described as a 'relative path', which adds meaning beyond the schema's bare type. However, it does not specify format, allowed characters, or examples, so the description only partially compensates for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the specific resource ('a file from the Card Builder media directory'), and the identifier ('relative path'). This distinguishes it from other card builder tools like card_builder_upload_media or card_builder_delete_card.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is given about when to use this tool versus alternatives, nor are any prerequisites or when-not-to-use scenarios mentioned. The description only implies that it should be used to delete media files, but lacks exclusions or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It states resources load on every page, but omits important details like whether adding duplicates overwrites or fails, permission requirements, side effects, or any confirmation of the action. This lack of depth leaves an agent unclear about the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two sentences that efficiently state the action and provide context with examples. No unnecessary words or repetition. Front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an add operation, the description covers basic purpose and scope (global loading). However, it lacks behavioral details like idempotency and response format. Since an output schema exists, the return value detail may be covered there, but the description should still hint at outcomes. Overall adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage for both parameters, so the schema already documents their types and defaults. The description adds no new meaning beyond the schema—it mentions 'JS module or CSS' which aligns with resource_type options but doesn't enhance understanding. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a custom resource (JS module or CSS) to Lovelace, establishing a specific verb and resource. However, it does not explicitly distinguish from sibling tools like dashboards_remove_dashboard_resource or dashboards_update_dashboard_resource, missing an opportunity to differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the tool, such as registering custom cards, themes, or scripts that load on every dashboard page. It implies usage scenarios but does not offer explicit guidance on when not to use it or mention alternatives like removal or update tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds minimal behavioral context beyond the operation itself. It says 'currently assigned' implying real-time data, but does not disclose read-only nature, permissions, rate limits, or any side effects. With no annotations, the description should provide more transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no unnecessary words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and an output schema, the description adequately covers the core purpose. However, it lacks usage context and parameter details, but given the low complexity, it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 0%, so the description must compensate. It mentions 'given area_id' but does not explain what area_id is, its format, or how to obtain it. This is insufficient for an agent to correctly supply the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists devices assigned to a specific area_id, which is a specific verb+resource. It distinguishes itself from the sibling tool 'devices_list_devices' which presumably lists all devices without area filtering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'devices_list_devices'. There is no mention of prerequisites, restrictions, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions 'Add' implying mutation, but it does not specify required permissions, side effects (e.g., does it install immediately?), or what the output schema returns. The behavior beyond adding is unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that effectively communicates the tool's action and key parameters. It is front-loaded with the verb 'Add' and resource 'custom HACS repository'. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool modifies HACS configuration and has an output schema (undocumented), the description lacks details on post-add behavior, error conditions (e.g., duplicate URL), and permission requirements. It is not fully self-contained for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning by listing category values (integration, plugin, etc.), but does not clarify the URL format or validation constraints. The description partially clarifies param usage but leaves gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: adding a custom HACS repository by URL and category. It lists the allowed categories (integration, plugin, etc.), which distinguishes it from sibling tools like hacs_install_hacs_repository or hacs_list_hacs_repositories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly conveys when to use (adding a custom repository), but it does not explicitly state when not to use or mention alternatives like hacs_install_hacs_repository for default repositories. No exclusion criteria or context for appropriate use is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'get details', which is a read operation, but does not mention any permissions needed, rate limits, or what the returned details include. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no filler. It could be slightly expanded, but it is well-structured and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to detail return values. However, there is no information on parameter semantics or usage context. The description is minimally complete for a simple read tool but lacks depth.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should compensate but only mentions 'by id' without explaining what the repo_id parameter represents or how to obtain it. The parameter is a simple string, but no additional context is given beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get details', the resource 'HACS repository', and the identifier method 'by id'. It effectively distinguishes this tool from sibling tools like hacs_list_hacs_repositories (list all) and hacs_install_hacs_repository (install action).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. It is implied that for single repository details this is appropriate, but there is no mention of prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description mentions 'fuzzy search' implying partial matching, but lacks details on idempotency, side effects, or auth requirements. No annotations are provided, so the description carries the full burden but only gives minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is highly concise and front-loads the key purpose. However, slightly too brief given the lack of parameter and usage details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Describes what is searched but omits output behavior (output schema exists but not referenced) and pagination context. With sibling search tools, more guidance would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, and the tool description does not explain the parameters (query, limit) or how the searchable fields map to them. The description lists searchable fields but does not clarify how to structure the query or limit results.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool performs fuzzy search across specific device attributes: name, name_by_user, manufacturer, and model. Differentiates from sibling search tools like search_search_areas and search_search_entities by targeting devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., search_find_related, search_deep_search). Does not mention when not to use it or specify prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states it sets volume with a range, implying a mutation. However, without annotations, it does not disclose whether the entity must be a media_player, any side effects, or prerequisites. The range hint adds some transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The single sentence is efficient and front-loaded with the action. Slightly more detail (e.g., entity type) could be added without harming conciseness, but it avoids redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, return values need not be explained. However, the description lacks context on entity constraints and fails to fully compensate for missing parameter descriptions. It is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate. It only hints at the volume parameter's range but provides no explanation for entity_id. This leaves a significant gap in parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'media player volume', with a valid range specified. This distinguishes it from sibling tools like services_set_climate_mode or services_media_play_pause.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., services_media_play_pause or other set_* tools). The description lacks context for appropriate usage or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states the prerequisite and action, but does not disclose key behavioral traits such as whether installation restarts services, if it is async or sync, error handling, or idempotency. This is minimal disclosure for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 11-word sentence that is front-loaded and efficient. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (one parameter, many siblings, no annotations), the description is incomplete. It does not explain the output, installation behavior, or how it differs from similar tools like supervisor_update_addon. The prerequisite is mentioned but could be expanded.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the single parameter 'slug'. The description says 'from its slug' but does not explain what a slug is or how to obtain it. It adds some context (prerequisite about repository) but does not compensate for the lack of schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool installs an add-on using its slug, with a prerequisite that it must be in a registered repository. This clearly distinguishes it from sibling tools like supervisor_uninstall_addon or supervisor_update_addon.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (to install an addon) and mentions a prerequisite but does not provide explicit guidance on when not to use it or how it differs from related tools like supervisor_start_addon or supervisor_update_addon. No alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full responsibility for behavioral disclosure. It only states 'stop + start' but fails to describe what happens during the restart (e.g., process termination, state loss, configuration re-read). This is insufficient for a safe operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence that front-loads the core purpose. Every word adds value, with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (1 param), the description is minimal but incomplete. It does not mention that the 'slug' parameter is required, nor does it provide any context about the return value or confirmation of success, despite the existence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter 'slug' with no description, and the description does not elaborate on what values are valid or how to find the slug. Schema coverage is 0%, so the description should compensate, but it adds no information about the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Restart an add-on (stop + start)' clearly states the action (restart) and the resource (add-on). It distinguishes itself from sibling tools like supervisor_start_addon and supervisor_stop_addon by combining both actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you want to restart an add-on in one atomic operation, but it does not provide explicit guidance on when to use this versus manually stopping then starting, nor does it mention any prerequisites or side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description only states the action without disclosing behavior such as duration, blocking nature, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, minimal words, no fluff. Clearly conveys the tool's purpose efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (no params, output schema exists), the description is fairly complete. However, it lacks differentiation from sibling tools for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so description does not need to add parameter semantics. Baseline 4 for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it creates a full Home Assistant backup. However, it does not distinguish from sibling tool 'supervisor_create_backup' which may have a similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like supervisor_create_backup. No context on prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It only states the file is deleted but does not mention whether the theme is automatically reloaded (given the `reload` parameter), what happens if the theme is active, or error conditions. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that immediately communicates the action. No unnecessary words. Efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete tool with an output schema, the description is adequate for the primary action but lacks details on the reload behavior and error handling. With two parameters and no schema descriptions, more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage. The description adds meaning to the `name` parameter by specifying the file format `themes/<name>.yaml`. However, the `reload` parameter is not explained, leaving its purpose unclear. Partial compensation but incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a theme file' and specifies the exact file location `themes/<name>.yaml`. This verb+resource combination distinguishes it from sibling theme tools like update or list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool instead of alternatives. It does not mention prerequisites (e.g., theme must exist) or the effect of the `reload` parameter. For a delete operation, some context about consequences would be helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description indicates a destructive action ('remove') but lacks details on side effects, error handling, or permission requirements. However, the basic behavior is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, compact sentence with no superfluous words. All content is relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating tool with no annotations and an output schema not described, the description lacks important context such as return value, idempotency, or typical error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for its two parameters. The description only mentions 'uid', leaving 'entity_id' unexplained, which is critical for identifying the todo list.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove') and the resource ('todo item from a list') using a specific identifier ('uid'). It effectively distinguishes from sibling tools like 'todo_add_item' or 'todo_list_items'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. Agents have to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided to declare behavior (e.g., readOnlyHint). The description only states a basic list operation without disclosing return format, pagination, or any side effects. For a read-only list, this is minimal, but the lack of annotations means the description should compensate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundant information. Every word contributes, and it is front-loaded with the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has zero parameters and an output schema is present, the description is sufficient for its simplicity. It clearly states the action and resource.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so schema description coverage is 100%. The description adds no parameter semantics because none are needed. Baseline 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'all devices in the device registry.' However, it does not differentiate from sibling tools like 'devices_list_devices' or 'devices_list_devices_in_area', which could cause confusion about the specific scope of this tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'devices_list_devices' or 'devices_list_devices_in_area'. There is no mention of context, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It states 'Auto-reloads automations afterwards', which adds transparency. However, it omits details on destructiveness, permissions, or side effects beyond the reload.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, concise and front-loaded. Every sentence adds essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and a destructive operation, the description should cover return behavior or error handling. It does not, and misses prerequisites or reversibility. Has an output schema but no mention of return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description adds value. 'by id' clarifies automation_id, and 'Set confirm=True to proceed' explains confirm's role. But it doesn't elaborate on automation_id format or restrictions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete an automation by id', specifying the verb and resource. It distinguishes from siblings like automations_delete_scene and automations_delete_script, but does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Set confirm=True to proceed', which implies a safeguard, but does not clarify when to use delete versus alternatives like disable. No explicit context for usage versus siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses one behavioral trait: auto-reloads scripts after deletion. But it omits crucial details like whether the deletion is permanent, what happens to automations referencing the script, required permissions, or confirmation behavior beyond setting confirm=True. Incomplete for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, front-loading the action and key behavioral notes. Every word is functional, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (delete script with confirm) and the presence of an output schema (not shown), the description covers the core purpose and a key behavioral note. However, it lacks details on side effects, reversibility, or return value, leaving gaps for an agent to fully understand the operation's context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'by id' for script_id and 'Set confirm=True' for confirm, but does not explain the format, constraints, or examples for script_id. The semantic value added is minimal, leaving ambiguity about valid values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete' and the resource 'script by id'. It also adds 'Auto-reloads scripts afterwards' to differentiate from similar delete tools. The purpose is unambiguous and distinct from siblings like automations_delete_automation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides the guideline 'Set confirm=True to proceed', which is a specific usage condition. However, it does not explain when to use this tool over alternatives like automations_disable_automation or how to ensure the correct script_id. Lacks explicit when-not or context for typical scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the action without disclosing side effects, permissions, or destructiveness. For a reload operation, important details like 'this may overwrite changes' are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, no wasted words, and front-loaded. It is as concise as possible for the information provided.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple with no parameters and an output schema, the description lacks completeness, e.g., it doesn't specify whether this is an admin-only operation or if it reloads from the default YAML file. Given no annotations, more context is warranted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% schema description coverage. The description adds no parameter information, but none is needed. Baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Reload scenes from YAML.' clearly states the action (reload) and the resource (scenes) with source (YAML). It distinguishes from sibling tools like automations_list_scenes (listing) and automations_reload_automations (reloading automations instead).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It is minimal and leaves the agent without context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It adds 'Auto-reloads scripts', indicating a side effect after writing. However, it omits crucial details: whether changes affect running scripts, if the operation is destructive (replaces entire config), error conditions, or permission requirements. The single behavioral hint is insufficient given the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences covering purpose and a key side effect. It is front-loaded with the core action. Every word earns its place with no redundancy. This is an exemplary model of brevity without sacrificing essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's mutation nature and available output schema, the description covers the basic action and auto-reload behavior. However, it lacks information on validation (e.g., what happens with invalid YAML), whether script must exist prior, or what the response contains. The output schema exists but is not described, so the agent is left guessing about return values. It is adequate but has notable gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, placing full burden on the description. The description adds that config is a 'YAML config (sequence, fields, ...)', which gives a vague hint but no specific parameter details (e.g., valid fields, format of config object, or how script_id is used). It does not compensate adequately for the missing schema descriptions. An output schema exists but is not referenced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create or overwrite a script YAML config', specifying the verb (create/overwrite) and resource (script YAML config). It mentions 'sequence, fields, ...' to clarify config contents. This distinguishes it from siblings like automations_set_automation_config (automation config) and automations_run_script (run, not config).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating or creating script configurations, but does not explicitly state when to use it versus alternatives like automations_get_script_config (read) or automations_run_script (execute). It lacks prerequisites (e.g., script existence) and exclusions. The context of siblings provides some inference, but the description alone offers no direct guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states 'delete an installed blueprint'. It does not disclose irreversibility, side effects (e.g., on automations using the blueprint), or permission requirements. The output schema exists but is not detailed, leaving behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No extraneous information; efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation, the description covers core purpose but misses parameter details (domain) and behavioral context (e.g., irreversibility, effects on dependent entities). Output schema exists but is not leveraged in description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. Description explains 'path' via example ('author/blueprint_name.yaml') but does not clarify the 'domain' parameter (default 'automation'). Only partial compensation for one of two parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (delete), resource (installed blueprint), and identifier (relative path with example). It effectively distinguishes from sibling tools like import, list, substitute by focusing on deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or when not to use this tool. The context suggests it's for deletion, but prerequisites (e.g., blueprint not in use) are not mentioned, and no alternatives are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses the main behavior (render and return dict) but omits details like whether it's read-only, error handling, or permissions, which are important for a tool with side-effect potential.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and efficient, containing no superfluous words while communicating the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 3 parameters, no schema descriptions, and no annotations, the description fails to clarify what the path refers to, what the input object should contain, or the possible values for domain. It is incomplete for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; the description only mentions 'given inputs' without explaining the path, input object structure, or domain parameter. It adds minimal value beyond parameter names in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool renders a blueprint with inputs and returns YAML as a dict, specifying the action and output format, distinguishing it from listing or importing blueprints.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for generating automation/script YAML from a blueprint, but provides no explicit guidance on when to use it versus alternative tools like blueprints_list_blueprints or blueprints_import_blueprint.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description should disclose behavioral traits like destuctiveness, idempotency, or auto-save. It does not. Only an example card_config is given, which provides no behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no waste. First sentence states the action clearly; second sentence provides a useful example. Could be slightly improved by front-loading the required parameters, but overall concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With sibling tools and no output schema details, description lacks context on whether this tool persists changes or requires subsequent save. The example helps but behavioral gaps remain. Not fully complete given the tool's role in a multi-step workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must explain parameters. It provides an example for card_config but does not explain url_path or view_index format/meaning. The example adds value but is insufficient for all three parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Add a card to a specific view in a dashboard.' Verb 'add' and resource 'card' with target are unambiguous. Distinguishes from sibling tool 'dashboards_add_view_to_dashboard' which adds a view, not a card.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. While the purpose is clear, there is no mention of prerequisites (e.g., requiring an existing dashboard and view) or relationship to save_dashboard_config. Implied usage from name but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full behavioral burden. It fails to disclose potential side effects, failure modes (e.g., if Puppet is not running), or response format details beyond the schema. Only states it captures a screenshot via Puppet.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences for the main function plus a bullet-style installation guide. Front-loaded with purpose. The installation steps are essential but make it slightly longer. Overall efficient with minimal fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the essential setup (add-on installation, token configuration) but lacks information about output format (despite output schema existing), error handling, or prerequisites like where to generate the long-lived token. Could be more complete for a tool with external dependencies.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds minor extra guidance for wait_ms ('Settle time... increase for heavy chart cards'), but otherwise relies on schema. Does not significantly enhance parameter understanding beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Capture a PNG screenshot of a Lovelace dashboard view via the Puppet engine.' The verb 'capture' and resource 'screenshot of a dashboard view' are specific. No sibling tool performs screenshots, so it stands out.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides detailed installation prerequisites for the Puppet add-on, which is necessary for usage. However, no explicit guidance on when to use this tool vs alternatives (e.g., other dashboard tools), leaving usage context implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses a side effect (creating grid source if needed), which is beyond the bare parameters. No annotations, so description carries the burden. However, it does not mention what happens if the source already exists, any required permissions, or details about the consumption flow itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No redundant words. Efficiently conveys the core purpose and a key condition (creating source if needed).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, return value info is not required. However, the description lacks details on parameter formats (e.g., entity ID patterns) and usage prerequisites. The simple nature keeps it adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 2 parameters with 0% description coverage. The description only hints at 'flow_from' but does not explicitly link to stat_energy_from or explain stat_cost. Missing parameter explanations force the agent to guess meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('Add'), resource ('grid consumption flow'), and context ('Energy Dashboard's grid source'). The condition 'creating the grid source if needed' adds specificity. Distinguishes from other energy_add_* tools by specifying 'grid consumption' and 'grid source'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like energy_add_battery_source or energy_add_solar_source. The description implies usage for grid consumption but doesn't exclude or compare against siblings, leaving the agent to infer selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must cover behavioral traits. It indicates mutation ('expose or hide') and the WS call, but lacks details on side effects, authorization needs, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, front-loading the purpose and necessary parameter constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists and the tool is a simple mutator, the description covers the main function. However, it omits any mention of entity existence validation or success/error behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. Only the assistant parameter is explained with valid values; entity_id and should_expose are not described at all.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (expose or hide) and the resource (entity to a voice assistant). It distinguishes from siblings like entities_get_entity_exposure and entities_list_exposed_entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lists valid values for assistant parameter and mentions the underlying WS call, but does not provide guidance on when to use this tool versus alternatives or mention prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It only gives format requirements but does not disclose behavior when multiple parameters are provided, default actions, or side effects like overwriting existing values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys purpose and format. It is front-loaded with the key action. Minor improvement could be adding structure for clarity, but it remains succinct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple set operation, the description covers the basic purpose and format. However, it omits details about parameter interactions and error cases. Since an output schema exists, return values need not be explained, but behavioral context is lacking.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds format hints for date, time, and datetime, which helps understand the string format. However, it does not explain the entity_id parameter or how the three optional parameters relate to each other.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'input_datetime', and provides specific format guidance (date, time, datetime), distinguishing it from sibling helpers for boolean, number, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs. alternatives. The tool name and date/time/datetime formats imply it's for input_datetime helpers, but no exclusion or comparison is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the types of relationships returned but fails to mention any behavioral traits such as performance characteristics, response size limits, or side effects. This is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that efficiently conveys the tool's purpose with no redundant words. Every part contributes to understanding the output, achieving high conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, presumably documenting the return format, so the description does not need to detail that. However, the description is brief given the complexity of returning multiple relationship types. It lacks context on ordering, pagination, or whether results are guaranteed to be non-empty.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the schema provides no descriptions for the single parameter (entity_id). The description does not elaborate on what entity_id represents or any constraints (e.g., format, required format). It adds no value beyond the schema itself.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool returns related entities (device, area, floor, sibling entities, area-mates) for a given entity, using a specific verb ('Return') and resource ('related entities'). It distinguishes itself from sibling search tools like search_search_entities or search_find_orphan_devices by focusing on relationship discovery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention any conditions, exclusions, or contexts where search_find_related is preferred over search_search_entities or other search tools. The description solely describes output, leaving usage ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions the filename path constraint but does not address file overwrite behavior, permissions required, side effects, or error handling, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with front-loaded main action, followed by a practical path hint. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no annotations and 0% schema description coverage, so the description should be more comprehensive. It covers basic function but lacks safety, error, and alternative usage context, making it incomplete for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for the 'filename' parameter (path inside HA config dir) and implies 'duration' is in seconds, but does not describe the 'entity_id' parameter format or that 'duration' has a default of 10 (only in schema). Given 0% schema description coverage, partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool records a short clip from a camera entity to a file for a duration, using specific verbs and resources. It distinguishes itself from sibling tool services_camera_snapshot by emphasizing 'clip' vs snapshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for recording video clips but does not explicitly state when to use this tool versus alternatives like services_camera_snapshot, nor does it mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. While the operation is likely read-only, this is not explicitly stated. The description does not disclose any side effects, permissions, or the nature of 'full info,' leaving behavioral traits ambiguous.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of a single sentence. It is front-loaded with the core purpose. However, it could be slightly more informative without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description does not need to explain return values. However, given the tool's role among many supervisor tools, additional context about prerequisites (e.g., ensuring the slug exists) or a link to list_addons would improve completeness. The description is adequate but lacks nuance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has a single 'slug' parameter with no description (0% coverage). The description adds value by clarifying that the slug identifies the add-on, which compensates for the missing schema description. The meaning is clear, though minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get), the resource (full info for an add-on), and the identifier (by slug). It effectively distinguishes from sibling tools like get_addon_logs and get_addon_stats by specifying 'full info'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as supervisor_list_addons (to get slugs) or get_addon_logs. The description lacks context on prerequisites or scenarios where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It lacks disclosure of side effects (e.g., does creation replace an existing pipeline? Are there permission requirements?). Only basic parameter guidance is given, not behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise: one line for purpose and one line for parameter guidance. It is front-loaded and contains no filler text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 11 parameters and no parameter descriptions in the schema, the description should cover more. It fails to explain required vs optional parameters, default behaviors, or return value. While an output schema may exist, the description itself is too sparse for a complex creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds value by explaining that engines are entity IDs with examples (e.g., 'stt.faster_whisper'). However, schema coverage is 0%, and only a few parameters are implicitly addressed; many like conversation_engine or language are left undefined. Baseline 3 with partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as creating a new Assist pipeline, with specific verb 'Create' and resource 'pipeline'. It distinguishes from siblings like voice_update_pipeline and voice_delete_pipeline by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions using list_services or list_entities to discover engines, which provides context for parameter values. However, it does not explicitly contrast with alternative tools like voice_update_pipeline, nor does it specify when creation is appropriate over updating.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. While 'Delete' implies a destructive action, the description lacks details on irreversibility, side effects, permission requirements, or error handling. The constraint about the preferred pipeline is helpful but insufficient for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two short sentences that convey the core action and a key constraint without any extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential usage constraint and the core purpose. While the output schema exists (so return values need not be explained), the description could still mention prerequisites beyond the preferred pipeline constraint, but it is minimally adequate for a simple delete tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter ('pipeline_id') with no description, and the tool description provides no additional meaning or context for this parameter. With 0% schema description coverage, the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete an Assist pipeline' and adds a specific constraint about the preferred pipeline, making it distinct from sibling tools like voice_create_pipeline or voice_update_pipeline.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states a condition for use: 'Cannot delete the preferred pipeline — change preference first.' This provides clear guidance on when not to use the tool, though it does not elaborate on alternatives beyond the implied prerequisite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the core behavior: it affects multiple entities. However, it omits details like atomicity, error handling, permissions required, or side effects. With no annotations, the description should provide more depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences with no extra words. It front-loads the main action and lists the possible action values. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of output schema, the description could have explained the response. It does not differentiate from sibling tools or mention scope limitations (e.g., domain must be known). More context is needed for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description lists possible values for 'action' but does not define 'area_id' or 'domain' beyond the default. Schema coverage is 0%, so the description should compensate but falls short. No parameter descriptions are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Turn on/off/toggle all entities of a domain in an area.' It uses specific verbs and identifies the resource (entities in an area by domain), differentiating it from sibling tools like entities_turn_on that target single entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for batch control of entities in an area, but does not explicitly state when to use this tool versus alternatives (e.g., entities_turn_on for a single entity). No exclusions or alternative tools are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description indicates read-only operation ('Get') but does not disclose what happens on error (e.g., invalid area_id) or any side effects, rate limits, or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is extremely concise (9 words, 1 sentence) with no unnecessary information. It effectively communicates the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, read-only), the description is mostly adequate. However, it omits mention of the output structure (even though an output schema exists) and does not clarify that 'states' implies a key-value mapping. Still, it's sufficient for basic understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description provides zero additional meaning for the 'area_id' parameter beyond its name and type. No format, examples, or constraints are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description is a single clear statement: 'Get current states of all entities in an area.' It uses specific verb and resource ('Get', 'current states'), and distinguishes from sibling 'areas_get_area_entities' which returns entity IDs, not states.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use vs alternatives, no exclusions or prerequisites. The description implies use when you need entity states in an area, but lacks comparison to similar tools or guidance on invalid area_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Minimal behavioral info: only that it enables. No annotations provided, but the mutation action is obvious from name and description. Lacks detail on side effects or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, concise, but too short to cover parameter meaning. Adequate but not optimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description lacks explanation of entity_id and output. For a tool with no annotations and low schema coverage, more detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 0%, description does not explain parameter 'entity_id' (what it represents). Fails to compensate for low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (enable) and the target (disabled automation). It distinguishes from sibling 'automations_disable_automation' which does the opposite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage: use when an automation is disabled and should be enabled. No explicit alternatives or when-not, but context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavioral traits. It only says 'reload all automations from YAML' without disclosing side effects, permissions needed, or whether it is safe to call repeatedly. The agent cannot assess the impact on running automations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words. It conveys the essential action efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an existing output schema, the description is minimally adequate. However, it could be more informative by explaining what 'reload' entails (e.g., re-reading YAML files and applying changes), but the current level suffices for a simple action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema_description_coverage is 100%. The description need not add parameter meaning. Baseline 4 for zero parameters is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'reload', the resource 'automations', and the source 'from YAML'. This clearly distinguishes it from sibling tools like 'automations_reload_scenes' or 'automations_reload_scripts'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as enabling/disabling automations or reloading scenes. There is no mention of prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It mentions persistence across restarts but lacks details on permissions, side effects (e.g., overwriting), or behavior on update. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, concise and front-loaded. Every sentence adds value with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and schema coverage is high, the description is adequate for a simple create/update tool. However, it lacks usage context and behavioral details, making it less complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters adequately. The description adds no extra parameter context beyond the tool's overall purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates or updates a Home Assistant entity group, specifying it wraps the group.set service and persists restarts. This distinguishes it from sibling tools like automations_remove_group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this versus alternatives like automations_remove_group or automations_list_groups. The description implies both create and update, but does not provide context for choosing this tool over others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states the tool lists events and uses a REST GET call, but fails to mention important aspects like whether events are sorted, pagination behavior, or any side effects. The output schema may cover return values, but the description itself offers minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences: the first states the core functionality, and the second provides the REST endpoint. There is no redundant information, and the important details are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the existence of an output schema (context signal shows true), the description is minimally adequate. However, it lacks usage guidelines and behavioral transparency, which would be needed for full completeness. It leaves gaps about time range inclusivity and sorting.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It specifies that 'start' and 'end' are ISO 8601 datetimes, which adds format context, but 'entity_id' is only identified as a calendar entity without further detail. This partial compensation earns a 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists events for a calendar entity within a given time range. It uses a specific verb ('List') and resource ('events'), and distinguishes from sibling tools like calendar_create_event (creates) and calendar_list_calendars (lists calendars).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving events in a date range, but provides no explicit guidance on when to use this tool versus alternatives (e.g., calendar_list_calendars for finding calendars, or calendar_delete_event for deleting). No exclusions or context are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the required format for SVG content and the return structure. However, it omits details about potential side effects (e.g., overwriting files), permissions, or constraints like file size limits, which would be helpful for an upload operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (under 100 words) and well-structured, front-loading the main purpose and then providing specific usage context and return value details. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description adequately explains the return shape. However, it lacks information on error scenarios, behavior if a file with the same name exists, and how the path parameter is used. As one of several upload tools, more contextual details would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning for svg_content by specifying the required format. The filename is mentioned in the return value but not described as a parameter. The path parameter is entirely ignored. This leaves two of three parameters undescribed, though the critical one is covered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (upload SVG) and resource (Card Builder media library). It explains the intended use case for AI clients designing SVGs inline. However, it does not explicitly differentiate from sibling upload tools like card_builder_upload_image_from_url or card_builder_upload_media, leaving room for ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that the tool is for AI clients designing SVGs inline, implying a context of custom, dynamically generated content. It notes the absence of preset styles. However, it does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives among the sibling upload tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description adds some behavioral context by specifying the internal endpoint and listing return fields. However, it does not explicitly state that this is a read-only, non-destructive operation, nor address potential performance implications of listing all devices.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with two clear sentences: one stating the main action and source, the other enumerating fields. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an existing output schema, the description covers the essential purpose. It lacks discussion of pagination or result size, but for a simple listing tool, it is largely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description cannot add value over the input schema. Per guidelines, baseline is 4 for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Tool name and description clearly state it lists all devices via a specific WS endpoint. While it doesn't explicitly differentiate from sibling tools like devices_list_devices_in_area or areas_list_devices, the 'all' qualifier distinguishes it as a broader listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., search_search_devices for filtered results or devices_list_devices_in_area for area-specific lists). The description lacks any context on appropriate usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the action without disclosing side effects (e.g., whether it overwrites existing sources, requires validation, or any destructive behavior). The description does not compensate for the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and parameter roles with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple add operation, the description covers the core purpose and parameter semantics. However, it lacks details about the output (despite an output schema existing), potential errors, and whether the action is idempotent. The minimal complexity earns a baseline adequacy score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameter descriptions (0% coverage), so the description adds crucial meaning by explaining that 'stat_energy_from' is for discharge and 'stat_energy_to' is for charge. This goes beyond the schema's raw string types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a battery source to the Energy Dashboard, specifying both charge and discharge measurements. It uses a specific verb ('Add') and resource ('battery source'), and distinguishes from sibling tools like energy_add_grid_consumption and energy_add_solar_source.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives or any prerequisites. It does not mention when not to use it or what conditions are required (e.g., existing Energy Dashboard).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, description carries burden but only states it shows last N lines and example content. Missing details like prerequisites, live vs static log, failure conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, verb and resource front-loaded, efficient for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple log tool with output schema. Could mention param optionality or behavior for clarity, but not critically lacking.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one param 'lines' with 0% schema description coverage. Description mentions 'last N lines' without explicitly linking to the parameter or explaining its optionality and default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Get', resource 'ESPHome add-on log output', and specifics 'compile errors, OTA status, device connections', distinguishing from sibling tools like supervisor_get_addon_logs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for debugging compile errors, OTA, etc., but no explicit when-to-use or alternatives mentioned. Lacks comparisons with other ESPHome tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose all behavioral traits. It mentions the automatic git-commit and rollback point, but omits prerequisites like an initialized git repo and potential side effects on the working directory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no redundancy; the key information is front-loaded in the first sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (git operations) and zero parameter coverage, the description is incomplete. It lacks details on how to use the parameters and preconditions, relying on the (unseen) output schema for return information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no information about the three parameters (relative_path, content, commit_message), leaving the agent without guidance on their semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Write a config file' and the unique feature 'automatically git-commit the current state before writing', distinguishing it from sibling tools like files_write_config_file and git commands.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description promotes this as 'the safest way to modify config files', implying use when safety is paramount, but does not explicitly state when not to use or mention alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the action without mentioning side effects, permissions, or state changes, leaving significant gaps for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single six-word sentence contains no fluff and is front-loaded. Every word is necessary and earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal but adequate for a simple parameterless tool. However, it lacks behavioral context (e.g., side effects) that would make it fully complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100% (implicitly). The description adds no parameter info, but baseline for 0 params is 4, and it does not detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Reload' and the resource 'all helper entities from configuration', making the purpose unambiguous. It distinguishes itself from sibling reload tools by specifying 'helper entities'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'system_reload_all' or other reload tools. The description lacks context on prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only says 'Fetch', implying a read operation without explicit safety or side-effect details. It does not disclose behavior beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of five words, perfectly concise with no wasted words. It earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description does not explain what the error log contains or provide any contextual details. It is adequate but minimal for a simple fetch.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description is not required to add parameter information. Baseline score of 4 applies for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch' and the resource 'Home Assistant error log', distinguishing it from sibling history tools like history_get_ha_config or history_get_logbook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like history_get_state_history or history_get_logbook. The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only states the purpose. It does not disclose any behavioral traits such as whether it is read-only, performance implications, or what data is returned beyond 'system/version info'. The burden falls entirely on the description, which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single, front-loaded sentence with no filler. Every word is meaningful and earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and the existence of an output schema, the description is minimally complete. However, it lacks detail about what 'system/version info' actually includes (e.g., version number, system name), which could aid an agent in interpreting results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is 100%. The description adds meaning by specifying that the tool retrieves system/version info, which is not evident from the empty schema. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Get') and the resource ('Home Assistant system/version info'). It is specific and distinct from sibling tools like history_get_ha_config or history_get_state_history, which deal with different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not specify context, prerequisites, or scenarios where this is preferred over other history or system tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the basic behavior (removal of a label) and the fact that other labels are preserved. However, with no annotations, it omits details like side effects, permission requirements, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that is front-loaded and contains no unnecessary words, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema and the tool's simplicity, the description provides adequate context for a removal operation. It could mention error cases (e.g., if the label or entity doesn't exist) but is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the tool description adds no additional meaning to the parameters 'entity_id' and 'label_id' beyond their names, despite the need to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove a label from an entity') and explicitly notes that remaining labels are preserved, distinguishing it from similar tools like 'labels_assign_label_to_entity' or 'labels_delete_label'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool versus alternatives (e.g., 'labels_remove_label_from_device'), nor does it mention prerequisites or contraindications.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. 'Fuzzy search' is mentioned but lacks details on case sensitivity, query length limits, performance, or output structure. With no annotations, description falls short.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that efficiently conveys the tool's purpose and scope. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 2 parameters, no annotations, and an output schema exists, the description is brief but does not fully explain output or edge cases. It covers the basic functionality but lacks completeness for a search tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It adds meaning by specifying search fields (entity_id, friendly_name, device_id) for the 'query' parameter, but does not explain the 'limit' parameter. Baseline is low due to 0% coverage, so score 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs fuzzy search on entities using specific fields (entity_id, friendly_name, device_id). Among sibling search tools like search_search_areas and search_search_devices, this one is distinct for entity-focused search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus other search tools (e.g., search_deep_search, search_find_unused_entities). The description implies entity search but lacks context on alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only says 'Reload configuration' without disclosing effects (e.g., disrupts running processes, requires authentication, reverts changes). The examples help but lack behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loaded with purpose and examples. No extraneous text. Efficiently conveys the core action and resource type.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description covers basic purpose but lacks behavioral details and clear differentiation from siblings. The output schema exists but is not leveraged to describe return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds meaning by listing example domain values (automation, script, scene, etc.), helping the agent understand what 'domain' refers to. However, it does not define the parameter explicitly or its expected format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reload configuration') and the resource ('domain'), with specific examples like automation, script, scene, input_boolean. This distinguishes it from sibling tools that target specific domains (e.g., automations_reload_automations), as this is the generic version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for reloading configuration of any domain, but it does not explicitly compare to sibling reload tools for specific domains (e.g., automations_reload_automations, helpers_reload_helpers). No guidance on when to use this vs alternatives, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully convey behavioral traits. It only states the basic render action without mentioning side-effects, error handling, authentication needs, or output format. This is insufficient for an agent to fully understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two efficient sentences. The first sentence front-loads the action and subject, and the second provides a specific use case. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool, the description is adequate but not thorough. It mentions output schema exists, so return values are not required, but it does not describe the output or any constraints. A mention of the output being a rendered string would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It adds context that the template is Jinja2 and uses HA engine, which goes beyond the raw schema type. However, it does not explain syntax, validation, or format details for the single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool renders Jinja2 templates using the HA template engine, and it provides a specific use case (testing). However, it does not explicitly differentiate from the sibling 'ws_render_template' WebSocket tool, which also renders templates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear when-to-use hint ('testing templates before saving'), but does not mention when not to use it or suggest alternatives. It implies a direct API call context versus WebSocket.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only states the basic action. It omits details about return values, error conditions, side effects (e.g., turning on/off the unit), or entity availability requirements. The existence of an output schema is not leveraged.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately conveys the action and the parameter values. It is free of superfluous words and follows a clear verb-resource structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 required string parameters, output schema exists), the description is minimally adequate but lacks details like error handling, device compatibility, or that entity_id must refer to a climate entity. It meets basic needs but leaves gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description adds meaning by listing the allowed values for hvac_mode (heat, cool, auto, off, fan_only, dry). However, it does not clarify what entity_id should be (e.g., a climate entity) or that the values may not be supported by all devices.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'climate HVAC mode', and lists the possible mode values (heat, cool, auto, off, fan_only, dry). It distinguishes itself from sibling tools like services_set_light_color or services_set_cover_position by specifying the domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as checking if the entity is a climate device or considering other climate-related services. It does not mention prerequisites or complementary actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits beyond listing. It does not state that the tool is read-only, nor does it mention any side effects, permissions, or output format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no unnecessary words. Front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and an output schema available, the description covers the basic purpose but omits details like whether it lists all backups globally or per-context, and lacks any filtering or ordering information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description correctly adds no parameter details. Baseline of 4 applies due to zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'all backups managed by Supervisor', distinguishing it from sibling tools like supervisor_create_backup, supervisor_delete_backup, and supervisor_restore_backup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It does not mention that it is read-only or that it should be used before creating or restoring backups.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose all behavioral traits. While 'DANGEROUS' hints at destructiveness, it does not explain consequences (e.g., overwriting current data, potential downtime, permission requirements). Critical details are missing for a dangerous write operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence plus a warning. It front-loads the core purpose ('Restore a full backup by slug') and immediately flags danger. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the dangerous nature of the tool, the description is notably incomplete. It does not mention prerequisites (e.g., looking up slug via list_backups), error states, or what the output schema contains. A more comprehensive description is warranted for such a high-risk operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for 'slug' (backup identifier) and 'confirm' (safety flag) but omits any explanation of 'password'. With three parameters, the lack of detail on one key parameter reduces value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (restore), the resource (full backup), and the identifier (slug). It distinguishes this tool from siblings like create_backup and delete_backup by specifying 'restore'. The added warning 'DANGEROUS' provides immediate context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states that confirm=True is required, signaling that this tool should only be used with deliberate intent. It does not list alternative tools or circumstances to avoid, but the danger flag effectively discourages casual use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool returns 'full state + attributes', implying a read-only operation. However, with no annotations, it does not explicitly state that it is non-destructive or mention any authentication needs or rate limits, leaving gaps in behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and output. Every word is meaningful, with no superfluous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description sufficiently covers the core functionality. However, it could mention error handling (e.g., if entity_id is invalid) or prerequisites (e.g., zone must exist), which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'entity_id', has no description in the schema (coverage 0%), and the tool description does not provide any additional context about its format, source, or required pattern. The parameter is self-explanatory by name, but the description adds no semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Get a single zone entity') and what it returns ('full state + attributes'), making it distinct from sibling tools like zones_list_zones.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., list_zones for multiple zones, update_zone for modifications). The usage context is only implied by the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses an important behavioral trait: 'only the fields you pass are sent', indicating a partial update rather than a full replacement. However, it does not mention other behavioral aspects such as whether the update is immediate, side effects, permissions required, or error handling. With no annotations provided, the description partially compensates but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the purpose and key behavioral detail. Every word contributes meaning, with no redundancy or filler. It is highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 7 parameters, zero schema descriptions, and no annotations, the description should provide more context. It does not explain the required entity_id format, the meaning of optional fields like passive or icon, or reference related endpoints (e.g., get_zone for current values). Even though an output schema exists, the description lacks essential usage details that an agent needs to correctly invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter descriptions are provided in the schema. The description adds the insight that only passed fields are sent, which affects how parameters are interpreted (incremental update). However, it does not explain the meaning of individual parameters like latitude, longitude, radius, passive, etc., leaving the burden entirely on the agent to infer from context. This is insufficient for a 7-param tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update'), the resource ('a zone'), and the mechanism ('via WS `config/zone/update`'). It specifies that only passed fields are sent, which distinguishes it from related tools like create or delete. This provides a specific verb+resource combination that is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating an existing zone (requires entity_id), but it does not explicitly state when to use this tool versus alternatives (e.g., create_zone for new zones, or delete_zone for removal). No when-not or prerequisites are mentioned. It provides basic context but no strategic guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is minimal and does not disclose any behavioral traits beyond the name. With no annotations, it fails to mention side effects, permissions, or output characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (4 words) and front-loaded with the essential information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with no parameters and an output schema, the description adequately conveys the purpose. It could mention that it retrieves all scenes, but it is generally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema coverage is 100% (empty object). The description confirms the tool takes no arguments, which is clear and sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'scenes', making the tool's function immediately apparent. It distinguishes from siblings like 'automations_activate_scene' or 'automations_list_automations'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., automations_activate_scene, automations_reload_scenes). The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutation ('Create or overwrite') and the side effect 'Auto-reloads automations', indicating immediate effect. However, without annotations, it lacks details on authorization, what happens to existing settings on overwrite, or error conditions. More transparency would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two short sentences that front-load the verb and resource. Every part adds value, with no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (nested config object, mutation side effects, output schema), the description is too brief. It omits parameter format details, prerequisites (e.g., automation must exist?), and behavioral guarantees. Sibling context suggests reading before writing, but this is not mentioned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to 'config' by listing example fields (alias, trigger, etc.), partially compensating for the 0% schema coverage. However, it does not describe 'automation_id' and the term 'YAML config' may conflict with the schema type 'object', potentially confusing agents about the expected format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create or overwrite' and the resource 'automation YAML config', and lists typical fields like alias, trigger, action, condition. This specificity, combined with the auto-reload note, distinguishes it from sibling tools like automations_get_automation_config or automations_delete_automation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies that this tool is for setting or updating automation configuration, but does not explicitly mention when to use it versus alternatives (e.g., reading before writing, enabling after configuration). No exclusions or prerequisites are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions the return payload ('empty payload on success'), which is helpful. However, with no annotations provided, it carries the full burden of disclosure. It does not discuss side effects, reversibility, or authentication requirements, leaving some behavioral uncertainty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two short sentences, no redundant information, and front-loads the key action. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with one parameter and an existing output schema, the description is adequate but lacks detail on parameter semantics and potential error states. It covers the basic purpose and return value but not the complete context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only says 'by id' without explaining the format, origin, or validation of the card_id parameter. The description adds minimal value beyond the schema's type declaration.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (a card by id). It distinguishes from sibling tools like create, update, and get cards, leaving no ambiguity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for deletion when a card_id is known, but it provides no explicit guidance on when to use this tool versus alternatives (e.g., update or disable). No prerequisites or error conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states return type and error possibility, but lacks disclosure on side effects, permissions, rate limits, or behavior for missing IDs. Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded, no superfluous information. Each sentence is valuable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter with one parameter and an output schema available, the description is nearly complete. It covers the basic functionality and return type, though it could mention error scenarios more explicitly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate. It clarifies that card_id is the identifier by saying 'by id', but does not explain what card_id values look like or how to obtain them. Adds some meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Get), the resource (card), and the return type (card dict with config or error). It distinguishes from sibling tools like card_builder_list_cards and card_builder_create_card.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when needing a single card's details, but does not explicitly mention when not to use it or suggest alternatives (e.g., card_builder_list_cards for all cards). No exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes the return structure and readiness for assignment, but does not mention behavioral traits such as idempotency (read-only), error behavior if the snippet does not exist, or any side effects. The description adds value beyond the name but lacks depth on non-obvious behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, both substantive: it states the action ('Fetch one snippet wrapped in the full target/container envelope') and explains the return format and usage. No filler or unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple fetch tool, the description covers the core function and output structure. However, it lacks preconditions (e.g., the snippet must exist), error handling, and does not mention that this is a read operation. With an output schema present but not visible, the description is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the description must compensate. However, it does not define what each parameter (name, target, container) represents. While parameter names and defaults provide some hint, the description adds no explicit explanation, failing to match the requirement for low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Fetch') and resource ('one snippet wrapped in envelope'), clearly distinguishing it from sibling tools like card_builder_list_style_snippets which list multiple snippets. It also explains the return format and how it's used (ready to assign to block's styles), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a single snippet for a specific target/container is needed, but it does not explicitly state when to use this tool over alternatives (e.g., card_builder_build_styles for building styles, or card_builder_list_style_snippets for listing). No exclusions or when-not-to-use guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden of behavioral disclosure. It only mentions listing files/folders, but does not explain how the path parameter works, whether listing is recursive, or if there are any side effects. For a read-only list operation, minimal info is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single, front-loaded sentence with no wasted words. It immediately communicates purpose and directory context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with an output schema, the description is largely adequate to distinguish from other listing tools. However, it lacks detail on parameter behavior and whether results are recursive, which could impact selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description adds some meaning by specifying the base directory. However, it does not explain the effect of the 'path' parameter (e.g., defaults to root, format) beyond the schema's default value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'files/folders in the Card Builder media directory', with the relative path given. This distinguishes it from sibling listing tools like card_builder_list_cards, which focus on different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for listing media files/folders, but does not explicitly state when to use it vs. alternatives like card_builder_list_cards or card_builder_list_style_presets. No guidance on when not to use or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states basic functionality. It does not disclose behavioral traits such as authentication needs, rate limits, error behavior, or what happens when the domain is invalid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and key features. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and 4 optional parameters, the description covers the main aspects: filtering, pagination, and field selection. It could mention when to use this tool over specific sibling tools, but is otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with detailed parameter descriptions. The tool description adds minimal extra meaning beyond the schema (e.g., 'optionally filtered by domain' and 'pagination'). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'list entities' with optional filtering by domain, pagination, and field selection. It distinguishes from sibling tools like entities_get_entity (single entity) and areas_get_area_entities (entities in area).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing entities with optional filters, but does not explicitly state when to use this tool versus alternatives like entities_get_entity or entities_list_entity_registry. No exclusions or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It discloses non-destructive and fast nature, but does not explain return value structure (though output schema exists) or behavior on validation failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no filler, front-loaded. Every word adds value. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no annotations, but with output schema, the description is minimal. Lacks prerequisites, parameter explanation, and error handling context. Adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. Description does not explain the single 'name' parameter, leaving its purpose (likely device or config name) ambiguous. Agent must infer from context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool validates ESPHome YAML config via Dashboard API, with 'no compile' differentiating it from compilation tools. Verb 'validate' and resource 'ESPHome YAML config' are specific and accurate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
'No compile, fast' implies use for quick validation rather than full compilation, but no explicit context on when to choose this over siblings like esphome_compile_device or esphome_write_config, nor any when-not-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the 'append' behavior but does not mention error handling, YAML validation (despite a validate_yaml parameter), or whether it creates backups. With no annotations, more detail would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and front-loaded with the action and an example. It is efficient but lacks some parameter details, earning a high score for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 0% schema description coverage, missing annotations, and 3 parameters, the description does not adequately cover parameter semantics or behavioral details (e.g., output, error cases). It is incomplete for agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 3 parameters with 0% description coverage, meaning the description must explain them. It only mentions 'content' implicitly via the example, but does not clarify 'relative_path' or 'validate_yaml' meaning. This is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Append content to an existing config file') and provides a concrete example ('adding an automation entry to automations.yaml'). This distinguishes it from sibling tools like files_write_config_file (overwrite) and files_read_config_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for adding content to existing files, not creating new ones. However, it does not explicitly state when not to use it or mention alternatives, such as files_write_config_file for new files.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose all behavioral traits. It only states that files are listed from the config directory or subdirectory, but omits details such as whether it lists files only or includes directories, hidden files, sorting, recursion depth, or any authentication/permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that conveys the essential purpose without extraneous words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one optional parameter and an output schema (which eliminates the need to describe return values). The description covers the core functionality but could be enhanced with details about file type filtering, directory traversal, or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'subdirectory' has no schema description (0% coverage). The description adds context by mentioning 'or a subdirectory', clarifying its purpose. However, it doesn't specify allowed path formats, separators, or behavior when the subdirectory doesn't exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List files') and the resource ('HA config directory'), explicitly mentioning optional subdirectory support. It distinguishes itself from sibling file operation tools (read, write, delete, etc.) by being a listing operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like files_read_config_file or git_git_status. Usage is implied by the nature of listing files, but no context about prerequisites, preferred scenarios, or exclusion criteria is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral details. It only says 'Create a new branch' without mentioning what happens if the branch already exists, whether it switches to the new branch, or any error conditions. The lack of detail leaves the agent uncertain about side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with 8 words, no redundancy, and includes both the action and a usage hint. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with an output schema (exists but not described), the description covers the basic purpose and a use case. However, it omits details like error handling, return value, and whether the branch creation switches context. It is adequate but leaves gaps that could affect correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a schema description coverage of 0%, the description should compensate but does not. It only rephrases the parameter name (branch_name) implicitly without adding format, constraints, or examples. For a single required parameter, the description adds no semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new branch' and adds context 'useful before experimental changes'. This distinguishes it from siblings like 'git_git_checkout_branch' which switches branches, and 'git_git_list_branches' which lists them. The verb and resource are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes 'useful before experimental changes', implying when to use the tool. However, it does not explicitly mention when not to use it, such as when a branch already exists, or compare it to alternatives like checking out an existing branch. Still, the guidance is clear enough for the intended use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It only states the action without disclosing behavioral traits like read-only nature, output format, or side effects. The description is insufficiently transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is clear and front-loaded. Every word is necessary and adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with an output schema, the description is mostly complete. However, it could mention that the output is a list of domain strings or what kind of results to expect, but since output schema exists, the lack of detail is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters (schema coverage 100%), so the description does not need to add parameter details. It correctly adds no extra parameter information, meeting the baseline for zero-param tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists integration domains that support config_flow, with a parenthetical explanation of config_flow. It distinguishes itself from sibling tools like integrations_list_flows_in_progress.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus other integration tools, such as integrations_list_flows_in_progress or integrations_start_config_flow. The agent must infer usage from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adds the important detail that existing labels are preserved. However, it does not mention idempotency, permissions, or other side effects. The simple nature of the operation mitigates this slightly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence that conveys the essential information without superfluous content. Front-loaded with the critical preservation note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool, the description covers the basic purpose. However, the lack of parameter explanations and usage guidance means an agent might need additional context to invoke correctly, especially without annotations or schema descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description does not explain what entity_id or label_id refer to or their expected formats. While the tool name provides context, explicit parameter guidance is missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Assign a label to an entity') and a key behavioral detail ('preserves existing labels'). It effectively distinguishes from sibling tools like 'labels_assign_label_to_device' by specifying 'entity'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when assigning a label while keeping existing labels, but provides no explicit when-to-use or when-not-to-use guidance. No alternatives are mentioned, though sibling tools exist for device assignments and label removal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden. It discloses that remaining labels are preserved, a key behavioral trait. However, it does not mention error handling (e.g., if label or device does not exist), permissions, or idempotency. More detail would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no superfluous words. It efficiently conveys the core action and a key preserving behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool with an output schema, the description is adequate but minimal. It covers the primary behavior but lacks details on error conditions, return values (though output schema may handle that), and permissions. It could be more complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage, and the description does not add any meaning to the parameters 'device_id' and 'label_id'. Although names are self-explanatory, the description should compensate for the lack of schema documentation but fails to do so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: remove a label from a device. It also specifies that remaining labels are preserved, which distinguishes it from bulk removal operations. The verb 'remove' and resource 'label from device' are explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for removing a single label while keeping others, but it does not provide explicit guidance on when to use this tool versus alternatives like 'labels_remove_label_from_entity'. No when-not-to-use or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states that the tool calls a service, which implies mutation but does not disclose idempotency, error handling, authorization needs, or side effects beyond the example. More detail on behavioral traits is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence plus an example, with no wasted words. It is front-loaded with the core purpose and immediately useful example, achieving maximum efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (handling return values) and many sibling tools for specific services, the description is largely complete for a generic caller. It could elaborate on scope (e.g., 'all domains and services') but the example covers the most common use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage. The description adds value by providing an example that illustrates the 'data' parameter format and how domain/service pair works. However, it does not explain all possible uses of the parameters, leaving gaps for non-light services.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Call any Home Assistant service.' It gives a concrete example (domain='light', service='turn_on') and the verb 'call' plus 'any service' distinguishes it from sibling tools that target specific services.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool can be used for any service but does not explicitly state when to use this generic tool versus specific siblings like services_set_light_color. No alternatives or exclusions are mentioned, leaving the agent to infer usage context from the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool returns CPU, memory, network, and IO stats. It does not state side effects, permissions, or whether it's read-only. With no annotations, the description carries the burden, but 'get' implies a read operation. Adding explicit safety notes would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the purpose and scope. No redundant words or information, front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple get-stats operation. It covers what stats are returned. However, it could be improved by noting whether stats are real-time or cumulative, and where to find the slug. The output schema exists (from context) so return format is covered externally.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 1 parameter ('slug') with 0% description coverage. The description does not explain what 'slug' is or how to obtain it for an add-on. Since schema coverage is low, the description should compensate but fails to add meaning beyond the schema's type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'runtime resource stats... for an add-on', listing specific stat types (CPU, memory, network, IO). It distinguishes from siblings like 'supervisor_get_addon' and 'supervisor_get_addon_logs' by focusing on stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. The usage is implied for retrieving stats, but no conditions or exclusions are mentioned. Sibling tools suggest other add-on operations, but the description does not guide the agent on when to pick this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully communicate behavioral traits. 'Validate' suggests a read-only check, but it does not explicitly state that the tool is non-destructive, what happens on success/failure, or any side effects. The presence of an output schema partially mitigates this, but the description lacks sufficient behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that efficiently communicates the core action and context. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is mostly complete. It clearly states the action and context ('before restart'), but could be enhanced by explicitly mentioning it is a pre-restart validation step.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and 100% coverage. Per guidelines, baseline for 0 params is 4. The description adds no parameter details, but none are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Validate' and resource 'Home Assistant configuration', clearly distinguishing this tool from siblings like 'system_restart_ha' or 'services_reload_config'. It is not a tautology and immediately conveys the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., configuration changes made) or that it should be run before restart. The context is only implied by the phrase 'before restart'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavioral traits. It states the tool is read-only ('reads update.* entities'), implying safety. However, it lacks details on permissions, rate limits, or behavior when no updates are available. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that front-load the main purpose and provide essential context. No wasted words; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (1 boolean param, output schema exists), the description is nearly complete. It explains what is listed and the data source. Slightly above baseline because it clarifies the entity model, but could mention non-destructiveness explicitly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description of the pending_only parameter. The description adds context about entities representing components, but parameter meaning is already fully covered by the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists updates (core, add-ons, HACS, custom components) and explains the underlying mechanism (reads update.* entities). This is specific and distinguishes it from sibling tools like hacs_list_hacs_critical_updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as hacs_list_hacs_critical_updates or supervisor_update_addon. No when-to-use or when-not-to-use advice is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the action (list) with no disclosure of side effects, permissions, or rate limits. For a read operation, minimal transparency is acceptable but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no unnecessary words. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal but sufficient for a simple list tool with no parameters. However, it could clarify what 'loaded' means or note that it lists config entries specifically.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters with 100% coverage. The description adds no parameter-level detail but is adequate since there are no parameters to describe. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all loaded integrations (config entries)' uses a specific verb ('List') and resource ('all loaded integrations'), and is distinct from sibling integration tools like 'get_all_integrations' or 'remove_integration'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'get_all_integrations' or 'list_flow_handlers'. The description does not mention exclusions or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fails to disclose behavioral traits beyond the obvious 'get' action. It does not mention read-only nature, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no extraneous words. Every word is necessary and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to detail return values. The tool is simple, and the description covers the essential action and input adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains that the 'name' parameter identifies the theme, compensating for 0% schema coverage. However, no additional constraints or format details are provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get', identifies the resource 'theme's variable map (CSS variables)', and specifies the means 'by name', clearly distinguishing it from sibling tools like themes_list_themes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose is implied, but there are no prerequisites or context hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not explicitly state that the operation is read-only or non-destructive. With no annotations, the agent has no safety context beyond the implied semantics of 'list'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that conveys the essential information without superfluous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with an output schema, the description covers the core purpose: listing all automations and their current state. No additional context is necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so the input schema provides complete coverage. The description adds no parameter info, but per guidelines, 0 parameters yields a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'automations', and adds 'with their current state' for specificity. Among siblings like automations_list_scenes and automations_list_scripts, it unambiguously refers to automations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as automations_enable_automation or automations_get_automation_config. The context for listing all automations is implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden but only mentions input flexibility. No disclosure of read-only nature, pagination, limits, or other behavioral traits beyond the minimal verb 'list'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action and resource. No wasted words; efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return details are not required. However, the description lacks context on what 'trace summaries' include, how many recent runs are returned, or any ordering. Adequate but with room for improvement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds meaningful constraint on acceptable formats ('bare id or 'script.<id>''), which goes beyond the schema's simple string type. This is valuable guidance for parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action 'List trace summaries (recent runs)' and object 'for a script', with added detail on accepted id format, distinguishing it from sibling tools like automations_get_script_trace.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for listing recent runs, but provides no explicit guidance on when to use this tool versus automations_get_script_trace or other siblings. Missing when-not-to-use indications.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only discloses the need for confirmation. It lacks details on what happens to entities in the group, permissions required, or reversibility of the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It efficiently conveys the essential action and requirement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with no annotations and an output schema, the description provides the minimum viable information. More context about the consequences of removal and the nature of entity groups would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the 'confirm' parameter by stating it must be True to proceed, which is not present in the input schema. The group_id parameter is already described in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it removes a Home Assistant entity group, using a specific verb and resource. It distinguishes itself from siblings like automations_set_group and automations_list_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage via the confirm flag but does not explicitly state when to use this tool versus alternatives such as automations_set_group or automations_list_groups.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description confirms it's a read-only operation returning the full config, but lacks details on what 'resolved' means, error handling, or permission requirements. With no annotations, the description carries the full burden and provides minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences: first states the action, second tells what to do with the result and offers an alternative. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to clarify the input parameters (0% schema coverage), leaving a significant gap. The differentiation from sibling tools is helpful but not enough to compensate for missing parameter guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description does not explain the 'name' or 'slot' parameters. The agent is left to infer their meanings from the tool name alone, which is insufficient for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the resolved DocumentData for a template without creating a card, and explicitly distinguishes it from create_card and make_template_card, providing specific verb+resource+scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context by offering alternatives: feed into create_card or use make_template_card to skip boilerplate. However, it doesn't explicitly state when not to use this tool or provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavior. It only states the action and output fields without mentioning side effects, prerequisites, or operational context (e.g., read-only, pagination).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loading the purpose, with no wasted words. Efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with an output schema and no parameters, the description is adequate but lacks behavioral context (e.g., read-only assurance, performance implications). Annotations could compensate but are absent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds value by hinting at output fields, but for an empty parameter set, a baseline of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all Card Builder cards' with a parenthetical list of returned fields, making the purpose specific and distinct from sibling tools like card_builder_get_card or card_builder_list_card_templates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. While the sibling context implies differences, the description lacks 'when to use' or 'when not to use' instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explains the return value and slot mappings but does not disclose any potential side effects, errors, or prerequisites (e.g., that the card must already exist). The behavior seems straightforward but could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient with three sentences, front-loading the purpose and output format. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and presence of an output schema, the description covers the return format and integration with dashboards_add_card_to_view. However, it omits prerequisites like creating the card first, and does not detail the slot_entities/slot_actions structures further.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining slot_entities and slot_actions with meaningful context ('maps each slot id to entity_id' and 'overrides action configs'). However, it does not elaborate on card_id beyond its name, which is somewhat obvious.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool builds a Lovelace card config for a Card Builder card and specifies the output format. However, it does not explicitly differentiate from other card_builder_* tools among the siblings, leaving some ambiguity about when to use this particular tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions using the output as a card in 'dashboards_add_card_to_view', which provides context, but it lacks explicit guidance on when to use versus alternatives, and does not mention when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description provides minimal behavioral context beyond the action; it mentions optional parameters but does not disclose idempotency, error handling, or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences: first states purpose, second lists optional parameters. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple turn-on tool, but lacks details on entity validation, error scenarios, and output expectation despite presence of output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates by listing optional parameters with ranges and formats (brightness, color_temp, rgb_color). Missing description for entity_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Turn on an entity' with a specific verb and resource, and distinguishes from siblings like entities_turn_off and entities_toggle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like entities_bulk_control or entities_turn_off. No context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It describes a read operation ('Get all HA entities'), which strongly suggests no destructive side effects. However, it does not disclose any required permissions, rate limits, or error behaviors (e.g., what happens if the device is not found). The description is adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 13 words, perfectly concise. It is front-loaded with the core action and delivers essential information without any redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and an output schema (provided but not shown). The description covers the basic purpose, but it does not mention prerequisites (e.g., device existence) or failure modes. Given the presence of an output schema, the return values are likely documented there. The description is minimally complete for a straightforward getter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, device_name, is described in the description as 'match by name/slug,' which adds useful context beyond the schema. However, since schema description coverage is 0%, the description could have provided more detail on the expected format or constraints. It adds some value but not enough to fully compensate for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: retrieving all Home Assistant entities that belong to an ESPHome device, with the matching criterion (name/slug) specified. It effectively distinguishes this tool from sibling tools like esphome_list_devices or entities_list_entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need entities for a specific ESPHome device, but it does not provide any guidance on when not to use it or how it compares to alternatives such as entities_list_entities. No explicit when-to-use or when-not-to-use information is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear the burden. It adequately indicates a read-only operation ('show') with no destructive effects. However, it does not disclose potential behaviors when the repository is uninitialized or empty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no extraneous words. Every word contributes to the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (one optional parameter, output schema present), the description covers the essential function and scope. It could mention the default limit or how the tool integrates with git, but output schema details are deferred.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description does not explain the 'limit' parameter, its purpose, or its default behavior. The parameter is left to the agent to infer from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Show recent git commits in the HA config directory' clearly states the action (show), the resource (git commits), and the scope (HA config directory). It distinguishes this tool from siblings like git_git_diff or git_git_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is for viewing commit history but does not explicitly state when to use it vs alternatives like git_git_diff or git_git_status. No exclusion criteria or specific use cases are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Warns that all changes after the commit are permanently lost, and explains that without confirm=True it returns a safety prompt. This is valuable since no annotations are provided, though it could detail side effects more.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first sets purpose, second gives warning and usage condition. No wasted words, front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive 2-param tool with no annotations, the warning and confirm explanation are good, but missing sha parameter details and output description (output schema exists). Adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds context: 'previous commit' relates to sha and 'confirm=True' explains the boolean. However, sha format and how to obtain it are missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb 'Hard-reset' and resource 'config to a previous commit', and the warning distinguishes this irreversible full reset from siblings like git_git_rollback_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this vs alternative Git tools (e.g., git_git_diff, git_git_rollback_file). The description only explains the confirmation flag, not use cases or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only says 'List all labels via WS ...' without mentioning side effects, permissions, rate limits, or scope (e.g., system-wide). For a read-only list tool, minimal transparency is acceptable but does not add value beyond the function name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. It is front-loaded with the core purpose 'List all labels'.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown) and no parameters, the description is minimally adequate. However, it lacks context about what 'all labels' entails (e.g., system-wide, user-specific) and does not mention alternatives or scope. For a simple tool, this is acceptable but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema description coverage is 100%. The description does not need to add parameter meaning. With no parameters, the description adequately conveys that no input is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all labels' clearly states the action (list) and resource (labels), distinguishing it from other label tools like create, delete, or assign. Mentioning the WS endpoint adds technical context but does not detract from purpose clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like labels_list_devices_with_label or labels_list_entities_with_label. However, the name and description imply it is for listing all labels, so usage context is implicit but not elaborated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides basic behavioral info (range and its meaning) but lacks details on side effects, error handling, or return behavior. It is functionally adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single concise sentence directly stating the action and value meaning. No redundant information; every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple command tool, the description covers the core action but misses context such as when to use tilt vs position, or behavior on invalid inputs. Given an output schema exists, return values need not be explained, but overall completeness is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the tilt_position parameter range (0-100) but does not document entity_id or any other parameter details. With 0% schema coverage, this partially compensates but is incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Set cover tilt position') and defines the value range (0=closed, 100=open), distinguishing it from sibling tools like services_set_cover_position which sets overall position.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through the range but does not specify when to use this tool over alternatives or prerequisites like cover tilt support. No explicit guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It does not disclose anything beyond purpose (e.g., read-only nature, permissions, limits). The brief statement implies a safe read operation but omits details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and result, no extraneous text. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with one parameter and an output schema (not shown). Description covers the basic purpose; output schema likely details return format. For a metadata retrieval tool, this is mostly adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description only says 'for specific statistic IDs' without explaining format, constraints, or examples of valid IDs. It adds minimal meaning beyond the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb "Get" and resource "metadata" and explicitly lists fields (unit, source, name). Clearly distinguishes from sibling tools like statistics_get_statistics which retrieve actual values.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly states when to use: to retrieve metadata for specific statistic IDs. Does not explicitly mention alternatives or exclusions, but the purpose is focused enough for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It warns 'DANGEROUS' and specifies the need for confirm=True, which is good, but lacks details on side effects, downtime, or what exactly gets destroyed. It is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences that convey essential information without any fluff. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter) and presence of an output schema, the description covers the basics. However, it does not explain what happens during the restart or provide any post-conditions, which could be useful for a dangerous operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one boolean parameter 'confirm' with default false, and the description adds critical context that it 'requires confirm=True', which is not evident from the schema alone. Schema coverage is 0%, so the description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Restart') and the resource ('Home Assistant Core'), and it is distinguishable from sibling restart tools like 'supervisor_restart_addon' and 'supervisor_restart_host'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it is dangerous and requires confirmation, but does not provide any guidance on when to use this tool versus alternatives, such as 'system_restart_ha' or 'supervisor_restart_host'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It mentions optional parameters and valid status values ('needs_action', 'completed'), which adds value. However, it does not describe the effect of omitting optional fields (they remain unchanged), nor does it mention return behavior or side effects. Some important traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. The first sentence states the action and optional params. The second clarifies status values. Every word is essential and the key information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers core purpose and optional params, but omits important contextual details for a simple update tool: what happens if both optional params are omitted, that the item must exist, and that `entity_id` identifies the todo list. An output schema exists but is not referenced. Missing behavioral context for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must add meaning. It explains `item` as new text and `status` with valid values, but does not describe `entity_id` or `uid` beyond identifying them. Partial compensation; would benefit from clarifying `entity_id` (e.g., which todo list) and that `uid` is the item identifier.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates a todo item by uid, with optional new text and status. This distinguishes it from sibling tools like todo_add_item and todo_remove_item, which handle different operations. The verb 'update' combined with resource 'todo item' provides specific action and scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the tool name and function, but there is no explicit guidance on when to use this tool versus alternatives (e.g., 'Use this to modify an existing item; use todo_add_item to create a new one'). No prerequisites or conditions are mentioned, so the guidance is only indirect.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that it fetches the most recent trace and explains the failed_only parameter's behavior. However, it does not mention any behavioral traits like rate limits, side effects, or return format (though output schema exists), making it adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with clear front-loading. Every sentence adds value: first states purpose, second explains a parameter. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema and sibling tools, the description covers the core use case adequately. However, missing description for automation_id and lack of explicit mention of what a 'trace' includes (though output schema may cover that) leaves minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains the failed_only parameter well but does not describe automation_id, leaving the required parameter undocumented. This partial coverage reduces the score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it fetches the most recent trace for an automation, using the verb 'fetch' and specifying the resource 'trace'. It distinguishes itself from siblings like automations_get_automation_trace and automations_list_automation_traces by calling itself a 'Shortcut' and focusing on the most recent trace.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context for when to use this tool: for debugging by fetching the latest failed trace. It implies the alternative of using list_automation_traces for other cases but does not explicitly state when not to use it or list all alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It indicates a read operation ('fetch') but does not disclose potential side effects, authentication requirements, rate limits, or error behavior (e.g., what happens if no trace exists or script_id is invalid). For a non-destructive read tool, the lack of such details is acceptable but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose and key parameter. It is front-loaded with 'Shortcut' to clarify intent. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema (likely defined elsewhere) and two parameters, the description is minimal but covers the core purpose. However, it lacks context on error handling, prerequisites, or how this tool relates to other trace-related tools. For a simple fetch tool, it is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the input schema has no descriptions. The description only partially compensates by explaining the failed_only parameter ('most recent failed') and implying that script_id identifies the script. However, script_id format or constraints are not described, and the tool's core parameter is left unexplained. This is insufficient for a tool with zero schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a 'Shortcut' to 'fetch the most recent (or most recent failed) trace for a script.' It specifies the verb (fetch), resource (trace for a script), and scope (most recent). This distinguishes it from siblings like get_script_trace (which fetches a specific trace by ID) and list_script_traces (which lists all traces).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use: when you want the most recent trace without specifying a trace ID, especially the most recent failed one via the failed_only parameter. It does not explicitly mention alternatives or when not to use, but the context of 'shortcut' and the parameter guidance is clear. For a simple fetch tool, this is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must cover behavioral traits. It does not disclose whether authentication is needed, what types of scripts are included (user vs system), or if there are any side effects. The description is too sparse.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three-word description is highly concise, front-loaded with the verb, and contains no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and presence of an output schema, the description is mostly complete. However, it could mention the scope of scripts (e.g., all scripts in the system) to avoid ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has zero parameters, so description adds no param info. Baseline for 0 params is 4, and description is adequate though minimal.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states 'List all scripts', which is a specific verb+resource combination. It clearly distinguishes from sibling tools like automations_list_automations and automations_list_scenes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like automations_list_script_traces or automations_get_script_config. Usage is implied by the name but not clarified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It fails to disclose behavioral traits such as read-only nature, authentication requirements, or error behavior. While listing is generally safe, the description is minimal and does not confirm safety or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that conveys the essential information efficiently with no extraneous words. It is front-loaded and earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, list operation) and presence of an output schema, the description is nearly complete. It could mention that it returns a list of blueprint objects, but the output schema likely covers that. Minor gap in explaining 'installed' semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description adds significant value by specifying the allowed values ('automation' or 'script') for the domain parameter. This clarifies usage beyond the schema's type and default alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and resource ('installed blueprints'), and specifies the domain parameter with valid values ('automation' or 'script'). It effectively distinguishes from sibling tools that list other resources (e.g., automations_list_automations).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context by mentioning the domain parameter, but does not explicitly state when to use this tool versus alternatives like blueprints_substitute_blueprint or automations_list_automations. No when-not-to-use guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It states it returns a 'compact list' and is non-mutating, but does not explicitly confirm it is read-only or safe. It also omits authorization or performance traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and no redundant words. It is efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main functionality and links to a related tool, but omits the include_internal parameter and does not mention pagination or limits. An output schema exists, but missing parameter documentation reduces completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero description coverage, so the description must fully document parameters. It explains the category parameter with valid values (basic, layout, entities, controls, advanced), but does not explain the include_internal parameter, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List every block type understood by Card Builder,' which is a specific verb+resource. It distinguishes from sibling tools like card_builder_get_block_schema by noting that this returns a compact list and the sibling provides full details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance: use the category parameter to filter, and call get_block_schema for full details. It implicitly tells when not to use this tool (when full props are needed), but lacks explicit when-not-to-use or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden of behavioral disclosure. 'List all' implies a read-only operation, but it does not explicitly state that it is non-destructive or elaborate on permissions or limits. This is minimal but acceptable for a simple listing tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It efficiently communicates the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with no parameters and an existing output schema, the description is mostly complete. It could clarify the scope (e.g., global vs. scoped) or mention that it returns a list of preset objects, but the output schema likely covers format details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and 100% schema coverage (empty schema). With no parameters, the description does not need to provide additional semantics. Per guidelines, baseline is 4 for 0 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all reusable style presets' clearly states the verb (list) and resource (reusable style presets). It distinguishes itself from sibling tools that create, delete, or update style presets, and from other list tools like card_builder_list_cards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as card_builder_get_style_preset or other list tools. The description does not mention any prerequisites or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the full burden of behavioral disclosure. It only states the tool lists dashboards, without mentioning any potential side effects, permissions, or limitations. The behavior is obvious but the description adds no additional context beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence that immediately conveys the tool's purpose. It is concise and front-loaded with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is nearly complete. It states the function clearly. A higher score might require mentioning whether it returns a list of dashboard IDs or objects, but the output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description does not need to explain them. Per guidelines, 0 parameters yields a baseline of 4. The description is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all Lovelace dashboards' clearly states the action (list) and the target resource (Lovelace dashboards). It is specific and distinct from sibling tools that create or modify dashboards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit usage guidance is provided. However, the simple listing action implies its use without needing alternative suggestions, and it is straightforward to infer when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description notes that the tool 'Overwrites existing config', a key behavioral trait. No annotations exist, so the description carries the full burden. It does not mention permissions, error handling, or whether changes are reversible, but the overwrite behavior is communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loading the main action and providing a clear workflow. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values are covered. The description provides essential workflow context (get then modify then save) but lacks info on potential errors or validation. It is minimally complete for a save tool with straightforward parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 2 parameters (config, url_path) with 0% schema description coverage. The description adds no meaning beyond implying config is the dashboard config and that url_path might identify a specific dashboard. Without schema descriptions, the description should explain these parameters but does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it saves and overwrites Lovelace dashboard config, and explicitly distinguishes itself from get_dashboard_config by recommending a workflow. The verb 'Save' and resource 'Lovelace dashboard config' are specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to use get_dashboard_config first, then modify and save. This provides clear when-to-use guidance, though alternatives are not exhaustively listed. It differentiates from siblings without being overly prescriptive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description implies read-only list operation but lacks any behavioral details beyond the obvious. No annotations to supplement, so minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with 9 words, front-loaded with verb, no waste. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with no parameters; output schema exists. Description covers the core action, though could clarify what 'critical updates' implies.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; schema coverage is 100% (trivially). Baseline score of 4 per criteria for zero-param tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists HACS repositories with pending upgrades, using specific verb and resource. Distinguishes from siblings like hacs_list_hacs_repositories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like hacs_list_hacs_repositories to get all repos.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It states the tool returns a 'full snapshot' combining states, devices, and entities, but does not detail the output structure, pagination, or performance implications. The presence of an output schema mitigates some burden, but the description lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that immediately conveys the tool's purpose and convenience. It is front-loaded with 'Convenience' and uses concise, jargon-free language. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, return value explanation is not necessary. However, the description does not clarify that the tool aggregates multiple queries (e.g., area states, area devices, area entities) or note any dependencies. The summary parameter is undocumented. Completeness is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate. It mentions area_id implicitly ('scoped to one area') but does not explain the 'summary' parameter (boolean, default true). The purpose of summary is unclear from the description alone, leaving the agent guessing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it provides a 'full snapshot scoped to one area (states + devices + entities)'. The verb 'get' is implied by context, and the resource is clearly the area snapshot. It distinguishes from sibling tools like 'snapshot_get_snapshot' (no area scope) and individual area tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description labels it as a 'convenience' tool, implying it simplifies fetching multiple data types for one area. It implicitly suggests using this instead of separate calls to areas_get_area_states, areas_list_devices, etc. However, it does not explicitly state when not to use it or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It discloses it is a read-only convenience wrapper that auto-discovers sum statistic IDs. However, it does not mention prerequisites (e.g., energy sensors configured), error conditions, or what happens if no sensors exist. The existence of an output schema partially rescues missing return format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The first sentence clearly states the primary purpose (verb+resource+scope), and the second adds valuable context (wrapper, auto-discovery, use case). Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only two parameters and a sibling tool for generic statistics, the description covers the core purpose but misses details about the period parameter and prerequisites (e.g., need for configured energy sensors). The output schema exists but the description could still reference expected results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no parameter descriptions in schema). The description explains 'days' implicitly via 'last N days' but does not explain the 'period' parameter at all. With 0% schema coverage, the description should have explained both parameters; it fails on period.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets sum statistics for all energy-related sensors (kWh, gas m³) for the last N days. It also distinguishes itself as a convenience wrapper around get_statistics, differentiating from the generic sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates it is a convenience wrapper around get_statistics and auto-discovers sum statistic IDs. It also mentions usefulness for EON energy consumption analysis, which implies when to use it. However, it lacks explicit guidance on when not to use it or when to prefer the generic get_statistics.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry full behavioral burden. It correctly implies a read-only operation by stating 'Get info', but it does not explicitly confirm safety, permission requirements, or potential error conditions. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. It efficiently conveys the tool's purpose and key return values.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, the description does not need to detail return values. It adequately covers the purpose and scope. It could mention availability prerequisites, but the tool is simple enough that this is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%. The baseline is 4. The description adds value by listing example output fields, confirming the purpose of the no-parameter call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves host OS information (e.g., HAOS version, hostname, kernel). This verb+resource combination is specific and distinguishes it from sibling tools like supervisor_get_core_info or supervisor_get_supervisor_info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives or any prerequisites. It does not mention that it is for host-level info only, nor does it exclude misuse cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that status filtering happens client-side, which is a behavioral trait beyond the schema. However, with no annotations, other behavioral aspects like pagination, limits, or return structure are not mentioned, leaving gaps for a listing tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences. The first sentence front-loads the purpose, and the second provides parameter details. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 parameters), presence of output schema, and the description covering the main intent and a key parameter behavior, it is mostly complete. The missing explanation for 'entity_id' is minor as it is implied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the 'status' parameter with typical values, but does not explain 'entity_id' as the todo list identifier. This adds some value but is incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists items in a todo list via a specific WebSocket endpoint, which is a specific verb+resource. It distinguishes itself from sibling tools like todo_add_item or todo_list_todo_lists by implying it deals with items within a list, not the lists themselves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing items but does not provide explicit guidance on when to use this tool versus alternatives like adding, removing, or updating items. It lacks when-not-to-use instructions or alternative suggestions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description should disclose behavioral traits like side effects or permissions, but it only states the basic function. It does not mention any constraints, required privileges, or what happens if no engines are installed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the primary action and resource. Every word adds value, and there is no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters) and the presence of an output schema, the description is mostly complete. However, it could briefly mention what kind of information the output contains, though the schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the description does not need to add any parameter details. The schema already covers everything, and the description is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (List), the resource (installed Text-to-Speech engines), and provides additional context that these are entities in the 'tts' domain. This distinguishes it from similar list tools like voice_list_stt_engines.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing TTS engines but does not explicitly state when to use it over alternatives (e.g., voice_list_pipelines or voice_list_conversation_agents). No exclusions or context are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It states 'list' implying read-only, but does not explicitly mention safety, auth requirements, or side effects. Adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundant words, front-loaded key information. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema (not shown but present), the description is sufficient. Specifies domain. Could mention it is read-only, but not strictly necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (input schema empty). Schema coverage is 100%. Baseline for 0 parameters is 4; no additional info needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists installed wake-word detection engines and specifies the domain 'wake_word'. Differentiates from siblings like voice_list_stt_engines which list different engine types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., other list tools or voice tools). No exclusions or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the WS endpoint and parameter format but does not disclose side effects, irreversibility, permission requirements, or error scenarios.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—one sentence—and front-loaded with the core action. Every word is necessary and no extraneous information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no nested objects) and the presence of an output schema (even if not detailed), the description covers the essential purpose and parameter format. It lacks information about the response or error handling, but is otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no description for the zone_id parameter (0% coverage). The description adds value by stating it accepts 'zone helper id or zone.<id>', clarifying acceptable formats beyond the schema's generic string type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete a zone') and the resource, and distinguishes it from sibling tools like zones_create_zone or zones_update_zone by specifying the delete operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to delete a zone) but provides no guidance on prerequisites, consequences, or when not to use it. No explicit alternatives or exclusion criteria are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not mention whether the tool is read-only, requires authentication, or has any side effects. The description only states what is returned, adding minimal additional behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that conveys the tool's purpose and output without unnecessary words, making it efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (which covers return values) and the simplicity of the tool (one parameter), the description is fairly complete. It lacks information about error handling or usage constraints, but for a straightforward location get it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds value by specifying that the parameter refers to a 'person.*' entity, clarifying the expected entity ID format beyond the schema's bare type string. This helps the agent understand what entities are valid inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns specific location data fields (latitude, longitude, gps_accuracy, source, state) for a 'person.*' entity, which is specific and distinguishes it from sibling tools like zones_get_zone or zones_list_persons_in_zone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for person entities only, but it does not explicitly state when to use it versus alternatives, nor does it mention any prerequisites or conditions. No sibling differentiation is provided beyond the entity type hint.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full behavioral burden. It correctly states the tool lists all areas, but adds no insight about permissions, side effects, or edge cases. For a simple list, this is adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no annotations, and an output schema present, the description fully covers the tool's purpose. No missing information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema description coverage, the schema already conveys everything. The description adds no parameter details but doesn't need to. Baseline per rubric is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'areas', clarifying they are 'rooms'. It distinguishes from sibling tools like areas_create_area and areas_delete_area by describing a distinct operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives or any prerequisites/exclusions. It solely states what it does without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, but the description is straightforward for a read operation. The presence of an output schema compensates for missing details about return values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an existing output schema, the description is complete for a simple list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description does not need to add parameter meaning. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and the resource (all floors), distinguishing it from sibling tools like areas_list_areas and areas_list_devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool compared to other list tools, nor any exclusion criteria or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must fully convey behavioral traits. It mentions the trace includes 'steps, timing, action results', but does not disclose side effects, authorization needs, or that it is a read-only operation. Lacks full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose, no superfluous words. Efficiently communicates the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, return value details are not needed. The description is mostly complete but could mention prerequisites like requiring a valid run_id from automations_list_script_traces.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage for 2 parameters. The description only vaguely refers to 'a specific run' for run_id, but provides no explanation for script_id or parameter formats. Inadequate compensation for missing schema metadata.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'full script trace', and specifies the scope 'for a specific run'. It distinguishes this from similar sibling tools like automations_get_automation_trace by focusing on scripts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context with 'for a specific run', indicating when to use the tool. However, it does not explicitly state when not to use it or mention alternatives like automations_list_script_traces for listing available run IDs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It details specific checks performed (state trigger, mode, etc.) and the return structure, implying a read-only analysis. However, it does not explicitly state that the operation is non-destructive or mention any required permissions or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with purpose and provides a bulleted list of checks and a structured return format. It is well-organized but somewhat lengthy; some details might be redundant if the output schema were documented.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that no annotations exist and the schema provides no descriptions, the description covers purpose, usage, parameter meaning, and return format extensively. It is missing an explicit statement about the tool being read-only and does not mention any prerequisites, but overall it provides sufficient context for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds that the tool accepts a single automation dict or a list of automation dicts. However, the schema shows a single string parameter 'yaml_content', creating ambiguity about whether the input should be a YAML string. The description would benefit from clarifying that the parameter is a YAML-formatted string.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates automation YAML against HA best practices, using a specific verb ('validate') and resource ('automation YAML'). It distinguishes itself from siblings like 'automations_validate_automation_references' by specifying it is a static linter, not a reference checker.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for static linting of automation YAML, but does not explicitly state when to use this tool versus alternatives like 'files_validate_yaml_content' or 'automations_validate_automation_references'. There are no exclusions or guidance on prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must cover behavior. It explains the config format (DocumentData v3) and the recommended workflow, but does not disclose side effects, immediate persistence, authorization needs, or error handling. The output schema partially compensates for return value details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (three sentences) and front-loaded with the core purpose. While efficient, it could add a bit more context about other parameters without becoming verbose. Currently, every sentence is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, nested config), the description adequately covers the critical 'config' parameter and the workflow, but fails to mention the required 'name' parameter or optional parameters. The output schema helps with return values, but overall completeness is moderate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning to parameters. It only explains the 'config' parameter (full block tree), leaving 'name', 'description', 'tags', 'categories', and 'author' unexplained. This is insufficient for a 6-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new Card Builder card', which is a specific verb and resource. It distinguishes from sibling tools like 'card_builder_update_card' by focusing on creation, and even refers to related tools for building and validating config, reinforcing its unique purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: use 'build_from_recipe' to construct the config, validate with 'validate_config' before saving, and refer to 'recipe_guide' for details. This clearly tells the agent when to use this tool versus alternatives and what preparatory steps to take.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavioral traits. It only states the action (list) with no mention of permissions, side effects, or read-only nature. Minimal information beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with parenthetical clarification. No wasted words, front-loaded with the action and resource. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only list tool with an output schema, the description sufficiently explains the tool's purpose. It is complete given the tool's simplicity and the presence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and the description does not add parameter-specific details. However, schema coverage is high (100%), so the description is not required to compensate. A baseline of 3 would be too low given the simplicity; a 4 reflects that no additional parameter info is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists CSS custom properties, specifically `@property` registrations. Distinguishes from sibling tools like card_builder_list_cards by specifying the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. While the sibling set includes create/delete variants, the description does not mention when listing is appropriate or how it relates to other list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies a read-only operation but does not mention permissions, error cases, or side effects. Lacks detail on behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description need not detail return values. It mentions 'Full description (docstring)' which covers the key purpose. Sibling context is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (name) with 0% schema description coverage. Description does not explain the parameter, relying on the context of 'single tool' which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the full description/docstring of a single tool and distinguishes itself from sibling discover_tool_search by recommending it be called after tool_search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance to call after tool_search to read details, establishing a workflow. Does not explicitly mention when not to use, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only states the tool returns info via a WebSocket endpoint, but omits details like idempotency, side effects, or access requirements. The brevity leaves significant ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, directly stating the purpose and the underlying endpoint. It is optimally concise for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema, the description is largely sufficient. It identifies the resource and provides examples. However, it lacks mention of prerequisites (e.g., Energy Dashboard setup) and could note that the result is a snapshot, not a stream.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With no parameters, the description adds value by enumerating typical return contents (cost sensors, currency), which the input schema (empty) cannot convey. This exceeds the baseline of 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns Energy Dashboard info, specifies examples (cost sensors, currency), and mentions the underlying WS endpoint. This effectively differentiates it from sibling tools like energy_get_energy_prefs or energy_add_*.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving Energy Dashboard info but provides no explicit guidance on when to use this versus alternatives (e.g., energy_get_energy_prefs). No when-not or exclusion criteria are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden. It only states it returns preferences, which is a simple read. It doesn't disclose any edge cases or side effects, but for a straightforward getter, it is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence of 14 words. It front-loads the key information and contains no redundant text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is a simple getter with no parameters and an output schema. The description tells the agent exactly what it returns, and the output schema covers the rest. This is complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description adds no parameter info. Since schema coverage is 100%, the baseline is 4. The description correctly reflects that no inputs are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Return the full Home Assistant Energy Dashboard preferences', using a specific verb and resource. It also includes the underlying WebSocket command, making it unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like energy_get_energy_info. No 'when to use' or 'when not to' context is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses blocking behavior and approximate runtime, and mentions the API used. With no annotations, this is adequate but could note side effects like file creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, front-loaded with purpose. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, blocking, duration, and API. Missing prerequisites like valid config or device existence, but output schema likely fills gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single 'name' parameter is not explained beyond the context of 'device'. Schema coverage is 0%, so description should clarify what 'name' refers to (e.g., device name, config name).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (compile), resource (ESPHome firmware), method (Dashboard API), and blocking behavior. Differentiates from siblings like validate or upload.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides expected duration (60-120s) implying it's a heavy operation, but does not explicitly state when to use vs alternatives like esphome_validate_config or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It discloses the file path (/config/esphome/) but does not mention behavior if file missing, permissions needed, or return format beyond implicit YAML.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no filler, front-loading the core action and path.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite an output schema, the description doesn't mention return value structure (e.g., raw YAML string). For a simple read tool this might suffice, but could be more complete given lack of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, but the description adds meaning to the sole 'name' parameter by noting it can include or omit '.yaml'. This clarifies input format beyond the schema's bare string type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads an ESPHome device YAML config from a specific directory, with a verb ('Read') and resource ('YAML config') that distinguishes it from siblings like esphome_write_config or esphome_validate_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives one usage hint about passing name with/without .yaml, but lacks guidance on when to prefer this over other ESPHome tools like esphome_list_devices or esphome_get_device_entities, and no prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the tool validates client-side but does not describe behavioral details such as what happens on validation failure, whether it is purely read-only, or any side effects. Basic behavioral traits are implied but not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the main purpose, and includes a helpful usage hint. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple validation tool with one parameter and an output schema, the description covers the core functionality and usage context. However, it omits documentation for the parameter and could briefly mention the nature of the output (e.g., error list). Still largely complete given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, device_name, is not described. Schema description coverage is 0%, and the description fails to add meaning to this parameter (e.g., what device_name refers to, expected format). This is a significant gap for a single-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it validates LVGL config client-side, specifically checking unique widget/page IDs and valid page references. It distinguishes itself from the sibling tool esphome_validate_config, which does full component/property validation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool (as a pre-check before compiling, runs without ESPHome Dashboard) and when not to (for full validation, use esphome_validate_config). This provides clear alternatives and context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes the operation as staging and committing, but does not disclose edge cases (e.g., no changes), behavior, or side effects beyond the obvious. No annotations to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, front-loaded with action. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with output schema present; description covers core usage and context. Could mention prerequisites or error handling but sufficient for typical use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 1 parameter (message) with 0% description coverage. Description only says 'with a message,' adding no detail on format, constraints, or usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it stages all changes and commits with a message. Distinguishes from siblings like git_git_checkout_branch and git_git_rollback_to_commit by focusing on committing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using 'before making risky changes as a checkpoint.' Does not mention alternatives or when not to use, but guideline is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly states it is a read-only operation ('without advancing'), indicating no side effects. However, with no annotations, it could further note that it does not modify any state, but the current phrasing is sufficient for transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise – one sentence that front-loads the verb and resource. No unnecessary words, and every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and a single parameter, the description is mostly complete. It lacks parameter description, which is a minor gap, but otherwise covers the tool's purpose and behavior adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description does not explain what 'flow_id' is or provide any additional meaning beyond the schema. This is a significant gap for a required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the current state of a config flow (without advancing)' clearly states the verb 'Get' and the resource 'config flow state', and explicitly distinguishes from advancing, which differentiates it from siblings like submit_config_flow_step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'without advancing' implies it is for inspection only, but there is no explicit guidance on when to use this versus alternatives (e.g., list_flows_in_progress, submit_config_flow_step). The usage context is implied but not fully articulated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains the heuristic criteria but does not disclose other behaviors such as performance implications or whether the operation is read-only. Since no annotations exist, the description bears the full burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that conveys the essential logic without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and the presence of an output schema, the description sufficiently explains what the tool does. It covers the core heuristic, making it complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds no param info, which is acceptable. Baseline 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it finds unused entities based on specific criteria (unavailable/unknown >7 days or orphan with no device). It distinguishes from siblings like search_find_orphan_devices, which focuses on devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like search_find_orphan_devices or search_deep_search. The heuristic nature is implied but not elaborated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the destructive nature with 'DANGEROUS' and the required safety flag, but it does not elaborate on consequences such as whether data is permanently deleted or if the operation can be undone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. Every word serves a purpose: the action, a warning, and a parameter requirement. No unnecessary text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's dangerous nature and 0% schema description coverage, the description is minimal. It covers the safety aspect but lacks details on return values, prerequisites, or post-uninstall effects. An output schema exists but is not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It clarifies that confirm must be set to true, which is critical beyond the schema's default of false. However, it does not explain the slug parameter, which is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Uninstall an add-on', which is a specific verb+resource. It distinguishes itself from sibling tools like supervisor_install_addon or supervisor_start_addon by focusing on removal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly warns 'DANGEROUS — requires confirm=True', giving a clear guideline that the tool should only be used with the confirm parameter set to true. This effectively prevents accidental misuse, though it does not mention when to use alternatives like stop or disable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes the action but lacks details on side effects, permissions, or what exactly 'reload' entails beyond refreshing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource, no wasted words. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal but sufficient for a simple reload tool with no parameters. Could mention effect on active theme or that it applies changes, but not strictly necessary given existing output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100%. Description doesn't need to add param info, and it meets the baseline for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Reload' and resource 'themes from <config>/themes/'. Distinct from sibling tools like themes_list_themes or themes_create_theme.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Call after creating or editing theme files.', providing clear context for when to invoke the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It implies a read-only list operation without side effects, but does not explicitly state safety, authentication needs, or result behavior. It is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that conveys the essential purpose without any extraneous words. It is perfectly sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (no parameters, output schema exists), the description is nearly complete. It could mention that it is a read-only operation, but that is implied. No major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%. The description adds value by specifying that the list is for the conversation step, which is beyond the schema. No parameters need explanation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (list) and resource (conversation agents), and specifies the scope (available for the conversation step of a pipeline). This distinguishes it from sibling tools like voice_list_pipelines or voice_list_stt_engines.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention any prerequisites or context. Given many sibling tools, this is a significant gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description holds the full burden. It discloses the basic action (setting a default) but does not mention potential side effects, required permissions, or error handling. For a simple mutation, this is minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that communicates the core purpose without extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (setting a default) and the presence of an output schema, the description is mostly complete. It could mention validation or effects, but overall it suffices.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one parameter (pipeline_id) with 0% description coverage. The description adds no meaning beyond the parameter name, failing to compensate for the low coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets the default Assist pipeline, using a specific verb ('Set') and resource ('default Assist pipeline'). It distinguishes from sibling tools like voice_create_pipeline and voice_get_pipeline.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes context that this pipeline is used when no pipeline is specified, implying when to use it. However, it does not explicitly state when not to use it or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool returns None if not found, which is useful, but does not mention permissions, side effects, or other behavioral traits. The read-only nature is implied but not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main purpose, and every word adds value. No redundancy or unnecessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (one param, output schema exists), the description adequately covers usage and behavior. It mentions return type and null case. Missing explicit error handling, but output schema likely compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description adds critical meaning: it specifies that the automation_id can be a bare id or prefixed with 'automation.', which is not obvious from the schema. This adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves an automation's YAML config as a dict, specifying the resource ('automation') and action ('get config'). It also distinguishes from sibling tools like 'automations_get_automation_trace' and 'automations_get_script_config'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the accepted input format (bare id or 'automation.<id>'), but does not explicitly state when to use this tool vs. alternatives or provide when-not-to-use guidance. Usage context is implied by sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It explains that the tool returns a dict and suggests a follow-up validation step, hinting it is likely read-only and transform-only. However, it does not disclose potential side effects (e.g., does it check schema or modify anything?), permissions needed, or error conditions. The level of disclosure is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient: it states purpose in one line, then provides a well-structured recipe example with clear comments, and ends with a brief output note. However, the example is somewhat lengthy and could be condensed or placed in a code block. Nevertheless, every sentence adds value, and the key info is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only one parameter, an output schema (not needed in description), and the description thoroughly documents the parameter shape and output usage, it is reasonably complete. It covers what the tool produces and how to use the result. It lacks information on error handling or edge cases, but for a transformation tool this is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only defines 'recipe' as an object with no additional constraints (0% coverage). The description fully compensates by detailing the exact shape of the recipe, including all keys ('slots', 'root_slot', 'root_entity', 'blocks') with inline comments and types. This provides complete semantic meaning beyond the schema, enabling correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies that the tool 'Materialise a recipe shorthand into a full DocumentData (config) dict'. This verb+resource combination is precise, and the context distinguishes it from siblings like 'create_card' (which takes the config) and 'validate_config' (which checks it). The recipe shape is provided, making the tool's function unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by showing the output should be passed to 'create_card' and suggests running 'validate_config' first. However, it does not explicitly state when to use this tool versus alternatives (e.g., directly constructing a config, using a template, or consulting a recipe guide). No exclusion criteria or contrast with siblings is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool makes HTTP requests to raw.githubusercontent.com and checks for schema matches. However, it does not mention side effects (likely none), authentication needs, error behavior, or rate limits. For a read-only check, this is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each adding value: first states the core action, second adds technical detail, third gives a use case. It is front-loaded with purpose and contains no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema, the description covers the main functionality and context. It mentions the external sources and the diagnostic purpose. A minor gap is not describing the output format (e.g., true/false or a diff), but the output schema likely fills that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is 100%. The description does not add parameter-level detail because none is needed. The baseline score of 3 is appropriate as per the guidelines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares the embedded schema with upstream Card Builder's HEAD manifest and block loader. The verb 'compare' and the specific resources (manifest.json, loader.ts) make the purpose unambiguous. It is distinct from other card_builder tools like 'card_builder_get_block_schema' or 'card_builder_list_block_types'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this when a recipe stops working — the upstream may have shipped a breaking schema change.' This provides a clear and specific usage context. While it does not list when not to use it, the guidance is sufficient for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses that the tool combines three sources and that the output is small, but does not detail any side effects, performance, or error behavior. For a non-destructive advisory tool, this is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is 3 sentences, each with a clear function: purpose, composition, and guidance. No fluff, well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description does not need to detail return values. It mentions the output contains pattern, template, and checklist. It also references optional parameters. It could clarify 'domain' and 'entity_id' further, but in context of Home Assistant card building, it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It adds meaning: 'free-text intent' clarifies intent is natural language, and 'optional domain/entity_id' indicates optional scoping. However, it does not explain what domain or entity_id represent beyond the names. Adds some value but not comprehensive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'recommend the best design pattern, the underlying template, and a checklist' given a free-text intent. It uses specific verbs like 'recommend' and 'combines', and distinguishes itself from siblings by referencing other tools for richer detail.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on when to use this tool and when to use alternatives: 'The reply is purposely small — pull richer text via design_principles() or get_design_pattern(name) if needed.' This implies this tool is for a quick recommendation, while other tools are for deeper dives. It does not explicitly state when not to use it, but the guidance is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond the schema: it notes that property names are camelCase and that anything else gets discarded by the CSS resolver. With no annotations provided, this extra detail helps the agent understand output constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences with zero waste. The key information (what it lists and the camelCase convention) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description sufficiently covers the list contents and a behavioral caveat. Minor omission: no mention of paging or sorting, but acceptable for a simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so baseline is 4. The description adds meaning by explaining what the output contains (categories with property names and unit conventions), which is useful beyond the empty input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists CSS style categories with property names and unit conventions. The verb 'list' and resource 'style categories' are specific, and it distinguishes from sibling list tools like list_style_presets or list_block_types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the description explains what it does, it does not indicate scenarios where this listing is needed or contrast it with other card_builder list functions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the tool lists snippets, implying a safe read operation. No additional behavioral details (e.g., return format) are given, but the description is adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences, each adding value: the first states purpose and usage, the second explains composition. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and the presence of an output schema, the description sufficiently covers what the tool does and how to use the results. It could mention the output format, but for a list tool it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, and schema description coverage is 100%. The description adds no parameter information, which is acceptable since there are none. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List built-in style snippets', indicating a specific verb ('List') and resource ('built-in style snippets'). It also explains how to use them (drop into styles or dz_styles) and compose with build_styles, differentiating it from other list tools like list_style_presets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage context: 'drop them straight into a node's styles or dz_styles' and 'Compose multiple snippets with build_styles'. However, it does not explicitly state when not to use this tool or compare to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly indicates this is a read-only validation (checks, returns ok/errors/warnings). It does not mention any side effects, permissions, or rate limits, but for a validation tool, the described behavior is sufficiently transparent. No contradiction with annotations (none provided).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence for purpose, one for checks and return format. No filler words, front-loaded with the key action. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, checks performed, and return format. Missing elements include when to use (usage guidelines) and any prerequisites or side effects, but for a simple validation tool with one parameter, it is largely complete. The presence of an implied output schema in the description helps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for the 'config' object, and the description adds context by calling it 'Card Builder config (DocumentData)' and listing internal checks (version, rootId, blocks). However, it does not detail the expected structure or properties of the config object, leaving room for ambiguity. The description adds some meaning beyond the bare schema but is not fully compensatory.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Lightweight structural validation for a Card Builder config', identifying the specific verb (validate) and resource (Card Builder config). Among siblings like card_builder_create_card or card_builder_update_card, this is the only validation tool, so it distinguishes well.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need to validate a config before creating or updating a card, but it provides no explicit guidance on when to use versus alternatives (e.g., other validation tools or simply testing directly). There is no exclusion or alternative mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It indicates destructive action via 'Remove' and a safety mechanism with confirm flag. However, it does not discuss side effects, irreversibility, or authentication requirements, which are moderately important for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The first sentence states the core purpose, and the second gives actionable advice. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with few parameters and an existing output schema, the description covers the essential action and usage hints. It lacks mention of return value or error handling, but the output schema likely covers that. Sibling tools provide context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning to resource_id ('numeric ID') and confirm ('to proceed'), but does not explain data types or constraints beyond the schema. The description provides basic clarification but not deep semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Remove' and the resource 'Lovelace resource', specifying 'by its numeric ID'. It distinguishes itself from sibling tools like dashboards_add_dashboard_resource and dashboards_update_dashboard_resource by being the removal counterpart.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides concrete usage guidance: 'Set confirm=True to proceed' and 'Use get_dashboard_resources() to find the resource ID'. It does not explicitly state when not to use, but it is clear enough for an agent to understand the action and prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses that this is an update (mutation) but adds no other behavioral details (e.g., whether updates are partial or full, permissions needed). The schema covers parameters adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the purpose. Two sentences with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and parameters are well-documented, the description is sufficient. It provides the core action and a helpful reference for obtaining the resource ID.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have schema descriptions (100% coverage). The description does not add additional semantic meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates the URL or type of an existing Lovelace resource, using a specific verb and resource. It distinguishes from sibling tools like add_dashboard_resource and remove_dashboard_resource by focusing on updating.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It advises using get_dashboard_resources() to find the resource ID, which is a helpful prerequisite. However, it does not explicitly state when to use this tool versus alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It mentions updating, partial updates, and a specific field constraint. However, it does not describe authentication needs, rate limits, or side effects. The reference to the WebSocket command adds some context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, each adding value: purpose, usage pattern, and specific parameter guidance. No wasted words, front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with an output schema, the description covers the essential purpose and one parameter. It omits possible error conditions or prerequisites (e.g., device must exist), but the schema implies device_id is required, so completeness is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It adds value for 'disabled_by' by specifying acceptable values. Other parameters are self-explanatory from names, and the instruction to pass only changed fields clarifies optionality, but more explicit guidance could help.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a device registry entry' which is a specific verb+resource. It distinguishes from siblings like 'devices_remove_device' and 'devices_list_devices' by focusing on updating an existing entry.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It instructs to 'Pass only the fields you want to change' and provides a specific constraint on 'disabled_by', implying partial updates. However, it does not explicitly state when not to use this tool (e.g., for deletion) or reference alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions the underlying WS call, which gives some tech context, but does not disclose authorization needs, idempotency, or whether exposures are additive or replaced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, each adding value. The purpose is front-loaded, with assistants default in the second sentence and technical detail in the third. No redundant or irrelevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a bulk exposure tool with an output schema, the description covers the key aspects: bulk operation, assistants default, and underlying mechanism. It lacks details on error handling or return effects, but given schema coverage, it is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains the assistants default and subset usage, but does not elaborate on entity_ids or should_expose beyond their names and types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Expose/hide many entities across one or more voice assistants in one call,' which is a specific verb+resource combination. It clearly distinguishes from sibling tools like entities_set_entity_exposure by emphasizing bulk operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states it's for many entities in one call and explains the default assistants with instructions to pass a subset. It implies when to use the bulk version over the singular one, but does not explicitly mention when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully carries the burden. It describes the operation and parameter constraint but does not explicitly state it is read-only or disclose any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no waste; it clearly states purpose and parameter constraint in a straightforward manner.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists and the tool is simple (one parameter), the description provides sufficient context for the agent to understand what the tool does and what inputs are valid.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description adds essential meaning by listing the three allowed values for 'assistant', which the schema does not constrain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists entities exposed to a given voice assistant, which is specific and distinct from sibling tools like entities_list_entities or entities_get_entity_exposure.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for querying exposed entities per voice assistant but does not explicitly guide when to use versus alternatives or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It states it shows diffs for uncommitted changes or a specific commit, but does not specify whether it shows staged vs unstaged changes or behavior for binary files. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 16 words, front-loaded with core functionality. No redundant information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 1-parameter tool with output schema, description covers the main use cases. However, it could be more precise about default diff behavior (e.g., against HEAD, staged vs unstaged), leaving minor gaps for advanced users.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage for the only parameter 'sha', the description adds crucial context: 'by SHA' explains its purpose and default behavior (omitted = uncommitted changes). This compensates well for schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool shows diffs of uncommitted changes or a specific commit by SHA. Verb and resource are specific, differentiating from siblings like git_git_status or git_git_log.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies when to use (when needing to see changes), but no explicit when-not-to-use or alternatives like git_git_status for file-level changes. Usage is clear from context but not formally guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the safety prompt and default commit behavior but does not detail side effects (e.g., working tree changes, no commit creation). It lacks permissions or idempotency info. Acceptable but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no waste. Information is front-loaded, clear, and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given complexity (3 params, git operation, output schema exists), the description covers key aspects: what it does, default, safety requirement. It omits finer points like file must exist in commit or output format, but for typical use it is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, so description must compensate. It adds meaning for sha (default HEAD = undo uncommitted) and confirm (must be true to proceed). But relative_path is not described beyond being a file path. Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool restores a single file to its state at a specific commit. It uses a specific verb (restore) and resource (single file). It distinguishes from sibling tools like git_git_rollback_to_commit (whole repo) and git_git_diff (show changes) by focusing on file-level rollback.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains default behavior (HEAD = undo uncommitted changes) and the safety mechanism (confirm=True to proceed). It implies when to use (file rollback) and not to use (whole repo rollback uses a different tool). Some explicit mention of alternatives would raise score to 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read-only operation with no side effects, but no annotations are provided to confirm. It does not mention authentication requirements, rate limits, or any potential impacts. For a list operation, the description is minimally adequate but lacks detail beyond what is obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the main action (list repositories) and includes the optional filter parameter. Every part is relevant and there is no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema (which presumably explains the return format) and the tool's simplicity, the description is largely complete. It could optionally mention pagination or limitations (e.g., number of results), but is adequate for its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero description coverage, but the description explicitly lists the valid category values, adding significant meaning beyond the schema (which only specifies string or null). However, it does not specify the exact format or case sensitivity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool lists HACS repositories with an optional category filter, and lists the valid categories (integration, plugin, theme, template, appdaemon, python_script). This distinguishes it from sibling tools like hacs_get_hacs_repository (single repository) and hacs_list_hacs_critical_updates (critical updates only).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing repositories with or without a category filter, but does not explicitly state when to use this tool over alternatives (e.g., hacs_get_hacs_repository for a specific repo) or when not to use it. No exclusions or guidance on prerequisites are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It correctly indicates a read operation but lacks details on authentication requirements, rate limits, or behavior if configuration is unavailable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence of 10 words that efficiently conveys the tool's purpose without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and zero parameters, the description is largely sufficient. It could mention that the output is structured as per the schema, but the current text is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is trivially 100%. The description does not need to add parameter details, and the baseline of 4 is appropriate due to no parameter complexity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'current Home Assistant configuration' with specific items in parentheses, distinguishing it from siblings like history_get_state_history or ws_get_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies using this tool to retrieve configuration but does not explicitly state when to use it over alternatives like ws_get_config or system_check_config. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the purpose and does not disclose any behavioral traits such as return format, permissions needed, or pagination behavior. Minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the action and resource, and contains no unnecessary words. Every part is essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has no parameters and an output schema exists, the description is sufficient to understand its function. It could mention that it returns a list of flows, but the output schema likely covers that. Complete enough for a simple listing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty (0 parameters, 100% coverage). The description does not add parameter information but the schema already fully defines the inputs. Baseline score of 4 for no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('config flows'), adding the filter 'currently in progress (started but not finished)' which clearly distinguishes it from sibling tools like get_config_flow or abort_config_flow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for viewing ongoing configuration flows without explicitly stating when not to use it or naming alternatives. Given the sibling tools, the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses creation/replacement behavior and linkage to dismissal, but doesn't elaborate on overwrite behavior or side effects. Adequate but not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, concise and front-loaded with the main action. No extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential behavior and linkage to sibling tool, but could include more context about persistence behavior or what happens without notification_id. Reasonably complete given simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. The description adds meaning for notification_id (linking to dismissal) but doesn't explain message or title beyond names. Partially compensates for lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create or replace') and the resource ('persistent notification') in the HA UI, and mentions the role of notification_id for later dismissal, which distinguishes it from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that providing notification_id enables dismissal via notify_persistent_dismiss, giving contextual usage guidance. It implies when to use, but doesn't explicitly state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states what is returned (health checks and system info) but does not disclose side effects, safety (read-only), authentication needs, or rate limits. Since it's a health check, it's likely safe, but this is implicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences that front-load the purpose and add relevant detail. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and the presence of an output schema (context signal), the description adequately explains the tool's return content (subsystems and system info). However, it lacks mention of any prerequisites or safety concerns, which would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and 100% schema coverage (empty schema). Per the baseline rule for 0 parameters, a score of 4 is appropriate as no parameter documentation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves system health status, listing specific subsystems (recorder, websocket, cloud, network) and basic system info (version, dev mode, virtual env). It distinguishes itself from sibling tools by focusing on health checks rather than configuration or updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for health monitoring but provides no explicit guidance on when to use this tool versus alternatives like system_check_config or system_ping_ha. No exclusions or usage context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description discloses it returns a JSON with themes and default themes, indicating a read-only operation. It does not mention any side effects, permissions, or rate limits, but for a list operation with no side effects, it is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states purpose, second provides return format. Both are essential and non-redundant. Well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and low complexity, the description is complete enough. Includes return format, which compensates for lack of output schema details. Could mention that it returns all themes in the registry, but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0 parameters, so schema description coverage is 100%. According to guidelines, 0 params baseline is 4. The description adds no parameter info, which is correct as there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all themes registered with the frontend, plus the active default theme.' It specifies the verb (list), resource (themes), and scope (all themes plus default), distinguishing it from siblings like themes_get_theme or themes_set_active_theme.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like themes_list_theme_files. However, the simple list nature and return format imply it's for retrieving the current theme configurations, not files. Sibling context helps, but description lacks when-not-to-use or exclusionary statements.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully convey behavior. It specifies the return format but does not explicitly state that it is a read-only operation or mention any permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words: one for purpose, one for return format. Efficiently conveys essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, obvious list operation), the description adequately covers its functionality and output. Could mention that it returns all pipelines, but the phrase 'List all' suffices.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100%. The description adds value by documenting the return structure (pipelines array and preferred_pipeline string), which is not part of the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all Assist pipelines and includes the preferred pipeline id, making the purpose specific and distinguishable from siblings like voice_get_pipeline.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool over alternatives such as voice_get_pipeline. Usage is implied but not compared to other pipeline tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it returns up to `count` events within `timeout` seconds, but does not describe blocking behavior, return format, or what happens on timeout/empty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. Parameters are integrated naturally.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the core behavior and parameters. An output schema exists, so return format is not needed. However, could clarify if call is blocking or async.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds meaning for all three parameters: event_type (any HA event type), count (up to), timeout (seconds).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Listen for any HA event type' with examples like 'zha_event', distinguishing it from sibling tools like ws_listen_state_changes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for listening to events and specifies count/timeout limits, but does not explicitly indicate when to use this vs alternatives (e.g., ws_listen_state_changes).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It explains the return behavior (up to count events within timeout seconds) and gives usage examples. However, it does not clarify if the call is a one-time query or a persistent subscription, nor disclose other traits like permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, front-loaded sentences. Every sentence adds value: the first states the core action and output, the second gives practical usage guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description covers the essential behavior and use cases. Since an output schema exists, it does not need to detail return values. It could mention more about the subscription nature (one-shot vs continuous) but is adequate overall.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains 'count' and 'timeout' with default behaviors, but does not describe 'entity_id' (though self-explanatory). The addition of usage context partially makes up for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Listen for' and the resource 'state change events for an entity'. It provides specific examples like watching a sensor or waiting for a motion trigger, which differentiates it from sibling tools like ws_listen_events (general events) and ws_subscribe_trigger (triggers).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes explicit use cases ('watching a sensor, waiting for a motion trigger, etc.') that indicate when to use the tool. However, it does not explicitly state when not to use it or mention alternatives, though the sibling context makes it implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the fallback behavior for filename derivation and states the tool downloads and uploads. It doesn't cover size limits, authentication, or error handling, but the key behavioral aspects are transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no fluff: first sentence states core operation, second gives use cases, third notes fallback behavior. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given sibling tools for media upload, the description covers key aspects. It could mention supported formats or access requirements, but the output schema (present) likely covers return values. Adequate for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It only hints at filename derivation from URL path but does not explain the 'filename' parameter's purpose or the 'path' parameter at all. Both optional parameters remain unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Download an image from any HTTP(S) URL and upload it to the Card Builder media library', specifying the exact action and resource. It distinguishes itself from sibling upload tools like card_builder_upload_media (likely for local files) by emphasizing URL-based download.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides two concrete use cases: when a marketplace card misses a background image and for reusing images from a public CDN. While it doesn't explicitly state when not to use it, the scenarios give clear guidance on appropriate contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It accurately describes the operation as a read (Get) with no side effects, but does not disclose any potential costs, authentication needs, or limits. The behavior is predictable but the transparency is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is appropriately sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential information for a no-parameter getter: what it returns (Lovelace resources). Since an output schema exists, the return format is handled externally, so no further detail is necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage, the baseline is 4. The description does not need to explain parameters, and it correctly describes what the tool does without redundant parameter info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'Lovelace resources', and specifies what those resources are ('custom cards, CSS'). This distinguishes it from sibling tools like dashboards_get_dashboard_config or dashboards_list_dashboards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives. While the purpose is clear from the name and description, there is no guidance on context or when not to use it. A minor improvement would be to mention that it's for retrieving available custom resources before adding cards.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states that adding a widget saves the config and that lambda callbacks are not supported, but lacks details on side effects like whether existing widgets are affected, or what happens if the page doesn't exist. The description gives some transparency but is not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is six sentences, front-loaded with the purpose, followed by parameter explanations and important usage notes. Every sentence adds value, and there is no redundant or irrelevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of adding an LVGL widget involving config changes and deployment, the description covers the main steps and limitations. However, it does not explain the output or return value (despite an output schema existing), and it omits error handling or validation details, leaving some gaps for complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description adds significant value by explaining widget_type with examples (label, button, slider, etc.) and properties with common keys (id, x, y, width, height, text, value, styles). This compensates well for the missing schema descriptions, though it doesn't cover all possible properties.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds an LVGL widget to a page on an ESPHome device and saves the config. The verb 'add' and resource 'widget' are specific, and the action distinguishes it from sibling tools like esphome_lvgl_delete_widget or esphome_lvgl_get_pages.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when not to use this tool for lambda callbacks, directing to use esphome_get_config/esphome_write_config instead. It also provides post-usage steps (validate, compile, upload), giving clear context on the workflow. However, it doesn't compare to other lvgl tools for scenarios like updating or deleting widgets.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that deletion triggers a multi-step process (save, validate, compile, upload) and that it is destructive. No annotations provided, so description carries the burden well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two succinct sentences with no wasted words. Front-loaded with the core action, then immediate follow-up steps. Ideal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers the deletion action and required follow-up steps. Lacks return value specification or error handling, but for a deletion tool this is acceptable given the simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage; description does not elaborate on parameters beyond their names. While names are self-explanatory (page_id, widget_id, device_name), the description adds no extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete an LVGL widget by id from a page on an ESPHome device', specifying the verb and resource, and distinguishes from siblings like add_widget or get_widgets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells the agent to run validation, compile, and upload after deletion, providing clear post-action steps. Does not mention when not to use, but context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It describes the operation as a read-only list and specifies the return values. However, it does not disclose potential errors, permission requirements, or dependency on a running dashboard, though these are less critical for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences with no unnecessary words. It front-loads the purpose and then specifies the output, achieving high efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and the presence of an output schema (though not shown), the description provides enough context about the tool's functionality and return values. It is complete for a simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so no parameter documentation is needed. The description adds value by explaining what the tool does, which is sufficient for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'ESPHome devices that have LVGL display support', and the output fields 'device names, page count, and page IDs'. It distinguishes from sibling tools like esphome_list_devices by specifying the LVGL filter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing LVGL-capable devices but provides no explicit guidance on when to use this tool versus alternatives like esphome_list_devices or esphome_lvgl_get_pages. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral burden. It discloses that the tool blocks until done (~30-60s) and uses OTA via Dashboard API. It also mentions network and password requirements. However, it lacks details on error behavior or side effects (e.g., device restart).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short, front-loaded sentences. Each sentence adds essential information: purpose, prerequisite, and blocking duration. No redundant words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a single parameter and an output schema (not shown, but stated). The description provides core information for using the tool: what it does, prerequisites, and blocking nature. For an OTA flash operation, it covers the most important aspects, though more detail on post-flash state could be included.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'name' is of type string with no schema description (0% coverage). The tool description implies 'name' refers to the device, but it does not explicitly define the parameter or clarify its format. Given the low schema coverage, the description should compensate but does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool does 'OTA flash compiled firmware to an ESPHome device via Dashboard API'. It specifies the action (flash), method (OTA), and target (ESPHome device). This distinguishes it from sibling tools like esphome_compile_device which compiles but does not flash.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides prerequisites: 'Requires device on the network and matching OTA password.' It implies the firmware is already compiled, suggesting prior use of esphome_compile_device. However, it does not explicitly state when to use or not use this tool, nor does it mention alternatives (e.g., serial flashing).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description states it is read-only ('without saving'), returns result/error, and handles custom tags. Sufficient for a validation tool; no contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no wasted words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, description adequately covers high-level behavior and return type. For a single-parameter validation tool, it is complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter 'content' with 0% schema description coverage. Description only says 'YAML content', adding minimal value beyond the parameter name. Could specify format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Validate', resource 'YAML content', and key distinction 'without saving'. Also mentions tolerance of Home Assistant custom tags, which differentiates it from other file tools. High clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions that the tool tolerates HA custom tags, indicating it's suitable for validating real HA config files. No explicit alternatives or when-not-to-use, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description provides key behaviors: YAML validation by default and auto-creation of parent directories. It does not specify overwrite behavior or error handling, but these are implied by 'write content'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information. The most important details (what it does, validation option, directory creation) are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers core behavior but omits return value, error conditions, and file overwrite behavior. Given the tool has an output schema (not shown), the description could be more complete. It's adequate but has gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has no parameter descriptions (0% coverage). The description adds meaning for validate_yaml (explains validation) but offers no additional semantics for relative_path or content. Compensation is partial; a 3 is appropriate given the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool writes content to a config file, validates YAML syntax, and creates parent directories. It distinguishes from siblings like 'files_append_to_config_file' and 'files_validate_yaml_content' by mentioning validation and overwriting behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on when to use validation (setting validate_yaml=False to skip), but does not explicitly contrast with alternative tools like append or write for non-config files. No when-not-to-use or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It correctly indicates a read operation (listing), but does not elaborate on any side effects or additional behavioral traits. For a simple read-only tool, this is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that instantly conveys the purpose. It is front-loaded with the key action and resource, with no unnecessary words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and has an output schema, the description is fully complete. It explains what the tool returns (list of orphan devices) without needing additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema is fully covered (100%). Per guidelines, with 0 parameters, the baseline score is 4. The description does not need to add parameter meaning, so this score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists devices that are orphans (no associated entities), using a specific verb 'list' and resource 'devices that have no entities associated.' This distinguishes it from sibling tools like search_find_unused_entities, which likely lists unused entities instead of devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when one wants to find devices without entities, but it provides no explicit guidance on when not to use or alternatives. Sibling tools exist for related searches, but no direct comparison is offered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description discloses return fields (id, name, source, unit_of_measurement) and filter behavior. No side effects mentioned, but the operation is clearly read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose. Example values add useful context. Could be slightly more structured but remains concise and effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description adequately explains return fields. For a simple listing tool, it covers essentials: purpose, parameter, and output format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one optional parameter with default 'sum' and no enums. Description adds meaning by explaining 'sum' vs 'mean' types and corresponding sensor types, compensating for 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List all available statistic IDs' and explains the context (sensor entities tracked by recorder). It distinguishes from siblings like statistics_get_statistics by focusing on listing IDs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for listing statistic IDs with optional filtering by type, but no explicit guidance on when to use this vs other statistics tools or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full weight. It discloses the destructive nature ('VERY DANGEROUS') and the confirmation requirement. However, it does not explain the full impact (e.g., service downtime, need for reconnection).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (10 words) with no unnecessary verbiage. It front-loads the action and follows with a critical warning and condition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple (1 param, output schema exists), the description omits what happens after reboot (e.g., system restart, agent must wait). It is minimally adequate but could improve with post-action context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must add meaning. It does: it tells that 'confirm' must be true for the tool to execute, which is critical and not obvious from the default false value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('reboot') and the target ('host machine'), leaving no ambiguity. It explicitly differentiates from sibling restart tools (e.g., 'supervisor_restart_core') by specifying 'host'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description warns 'VERY DANGEROUS' and requires 'confirm=True', giving clear guidance on when to use (only with confirmation). It does not explicitly list alternatives or when not to use, but the danger implication is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It states a simple reachability check, which is adequate for a ping tool. However, it does not specify what 'reachable' entails (e.g., response time, error codes) or whether it is non-destructive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no parameters and an output schema exists. For a simple ping operation, the description sufficiently explains the purpose and behavior, leaving no gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the schema coverage is 100% trivially. The description adds no parameter details, which is acceptable. Baseline score of 4 is appropriate per guidelines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Check if Home Assistant is reachable' uses a specific verb ('check') and resource ('Home Assistant'), clearly distinguishing it from sibling tools that perform other actions like controlling areas, automations, or integrations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for verifying connectivity but does not explicitly state when to use it compared to other tools, nor does it provide exclusions or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It indicates a read-only operation (list), which is accurate, but lacks details on response format or pagination. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loaded with the action verb 'List', immediately specifying the resource and domain. No unnecessary words, making it highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and presence of an output schema (which documents return format), the description is fully sufficient for this simple list operation. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist in input schema, so baseline is 4. Description appropriately mentions no parameters and provides the only relevant detail (domain scope).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists installed Speech-to-Text engines and specifies they are entities in the 'stt' domain, effectively distinguishing from similar sibling tools like voice_list_tts_engines and voice_list_wake_word_engines.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit instructions on when to use this tool versus alternatives, but the domain qualifier provides implicit guidance. Sibling tools serve different modalities (TTS, wake word), so context is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that it uses WebSocket and is faster, but does not detail any other behavioral traits such as rate limits, authentication requirements, or whether it is strictly read-only. For a simple zero-parameter tool this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the core purpose and adds a key differentiator. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists (so return values are documented), the description fully covers the necessary context: it gets all entity states, uses WebSocket, and is faster for large installs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, the input schema is fully covered. The description does not need to add parameter details; baseline score of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets all entity states via WebSocket, and explicitly differentiates by noting it is faster than REST for large installs, distinguishing it from sibling tools like entities_get_entity or history_get_state_history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (for large installs when speed matters) by contrasting with REST, but does not explicitly name alternative tools or provide clear when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only says 'list', which implies read-only but does not explicitly state safety, permissions, or side effects. For a simple list, it's minimally adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single concise sentence using code formatting. It is front-loaded and contains no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, description need not explain return values. It lists the attributes returned, which is sufficient for this simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; schema coverage is 100% trivially. Instructions give baseline 4 for 0 parameters. Description does not need to add parameter info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists all zone.* entities with specific attributes. It distinguishes from sibling tools like zones_get_zone (single zone) and zones_reload_zones (reload).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for listing zones but does not provide when-to-use vs alternatives or exclusions. No explicit guidance on when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description discloses the mandatory confirm parameter and the side effect of auto-reloading scenes. It does not mention irreversibility, but the deletion behavior is implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences with no wasted words. Every sentence adds essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with two parameters and an output schema, the description covers the critical usage requirement (confirm) and a side effect (auto-reload). It is adequate but could mention the tool's irreversibility.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description bears the burden. It explains that confirm must be True, adding value beyond the schema default. However, scene_id is not described, but its purpose is obvious from context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a scene config,' which is a specific verb and resource. It distinguishes the tool from siblings like automations_delete_automation and automations_delete_script, which delete different objects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes that confirm=True is required to execute the deletion, and that scenes are auto-reloaded after deletion. It does not explicitly mention when not to use this tool, but the context is clear for a straightforward delete operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that the tool attempts the call and may report 'not_implemented' if unavailable, adding important behavioral context beyond the basic action. No annotations present, so description carries full burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second adds critical caveat. No filler, front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core action and important limitation. Output schema exists but not described; for a simple delete tool, this is largely adequate. Minor omission: no mention of what happens on successful deletion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description explains that `uid` identifies the event, but does not clarify the role of `entity_id`. With 0% schema coverage, more parameter detail would be beneficial. Only one of two required parameters is explained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a calendar event by its `uid`', using a specific verb and resource. It distinguishes from sibling tools like calendar_create_event and calendar_list_events.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for deleting events, and adds caveat about HA not exposing a stable service, which helps set expectations. However, lacks explicit when-to-use vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return format (list of dicts with entity_id and name) and the REST method, which discloses behavior beyond mere read; no annotation provided, but description is sufficient for simple list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no extraneous information; includes REST path and return format efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and presence of output schema, description covers essential purpose and return structure; could mention potential pagination or filtering, but not critical for a list-all call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so baseline 4 is appropriate. Description does not need to add parameter info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List all calendar entities' with REST endpoint, distinguishing it from sibling tools like calendar_list_events which list events within a calendar.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage: use to get all calendars. Does not explicitly mention when to use vs alternatives or provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adequately covers what the tool returns (design patterns with specific fields) and the filtering option. It lacks details on the return format, pagination, or error handling, but is sufficient for a listing tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the core purpose, and every sentence adds value. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter) and that output schema exists, the description covers the essential aspects. It explains what each pattern contains, which is sufficient for an agent to decide when to use this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The domain parameter is explained in the description as a filter for HA entity domains, adding meaning beyond the schema's type description. Schema coverage is 0%, so the description compensates well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists curated UX design patterns for card building, explaining each pattern's content (intent, template, UX rationale, anti-patterns). It distinguishes from siblings like card_builder_get_design_pattern by being a list operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on using the tool, especially how to filter by domain. However, it does not explicitly contrast with alternatives like card_builder_get_design_pattern for retrieving a single pattern, leaving the agent to infer usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It discloses the search algorithm (BM25-lite), result structure, and namespace filtering. However, it does not explicitly state it is read-only or non-destructive, though implied. Adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the main action, no fluff. Clearly structured with purpose, mechanism, parameters, and usage tip.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description covers inputs, return format, and usage context. It doesn't discuss edge cases like empty results, but this is acceptable for a search tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains all three parameters: query, top_k (with default), and namespace (with example). This adds meaning beyond the schema types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'fuzzy search', the resource 'Nexus tool catalogue', and what it returns (top_k tools per query scored by BM25-lite). It distinguishes from sibling tools which are domain-specific (areas, automations, etc.) by positioning this as a discovery tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use this tool to discover the right tool name before calling it, avoiding memorization of ~250 tools. This provides clear context. Missing explicit 'when not to use' or alternatives, but the purpose is implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. It not only states the removal action but also reveals an important side effect: stripping matching grid `flow_from` entries. This adds transparency beyond the bare operation, though it omits details like irreversibility or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that efficiently conveys the core action and a key side effect. Every word adds value, with no redundancy or unnecessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, existing output schema), the description is largely complete. It covers the primary effect and an important secondary effect. It does not explain the output or provide usage context among sibling tools, but the output schema fills that gap. Overall, adequate for selecting and invoking the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains that the single parameter `stat_energy_from` identifies the energy source to remove, adding meaning beyond the schema which only specifies type and required status. However, it does not provide format examples or validation hints, so the value is moderately high but not exhaustive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove') and the resource ('energy source'), with the specific identifier `stat_energy_from`. It distinguishes this from sibling energy tools that add sources, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives, such as other energy or removal tools. The usage context is implied by the name and description but lacks explicit when-to-use or when-not-to-use instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses validation of YAML syntax, support for special tags, and file creation behavior. However, it does not mention overwrite behavior, error handling on validation failure, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no redundancy. First sentence immediately states verb and resource, followed by behavior and edge-case details. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given simple parameters and no nested objects, the description covers core aspects: purpose, location, validation, tag support, and creation. Missing details on overwrite and error responses, but overall adequate for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% but parameters are basic strings. The description adds context about the file path (/config/esphome/) but does not clarify what 'name' represents (e.g., device name vs filename) or if '.yaml' extension is needed. Minimal added value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Write' and the resource 'ESPHome device YAML config', with a specific path '/config/esphome/'. It distinguishes from siblings like 'esphome_validate_config' and 'esphome_get_config' by focusing on writing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context: writes to a specific directory, validates YAML, supports custom tags, and creates files. It does not explicitly state when not to use this tool or list alternatives, but the purpose is clear among esphome siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden. It clearly indicates a read-only listing operation with no side effects. No destructive or behavioral surprises; transparency is adequate for a simple list tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately conveys the tool's purpose. No wasted words; it is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description covers the purpose and scope adequately. It lists example helper types, providing useful context without needing more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description does not need to add parameter details. The baseline for zero parameters is 4, and the description is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all helper entities and provides specific examples (input_boolean, input_number, etc.), making it distinct from sibling list tools like 'entities_list_entities' or 'areas_list_areas'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for retrieving all helpers but offers no explicit guidance on when to use alternatives or when not to use this tool. Sibling list tools exist, but no comparison is made.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden of behavioral disclosure. It explains that the tool returns the first step (form, schema, errors) and provides an example output. It implies a stateful action by starting a flow, but does not detail side effects or prerequisites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with three sentences including an example. It is front-loaded with the core purpose and avoids extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's role in a multi-step config flow and the presence of an output schema, the description provides sufficient context to use the tool correctly. It links to the next step but could mention error handling or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for the 'handler' parameter by referring to 'integration domain' and giving an example ('shelly'). However, the 'show_advanced_options' parameter is not described, and schema coverage is 0%, so the description does not fully compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool starts a new config flow for a given integration domain, using specific verbs and resources. It distinguishes itself from sibling tools like 'integrations_submit_config_flow_step' by explaining the sequence of steps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the user to pass the returned 'flow_id' to 'submit_config_flow_step', providing clear context for the next step. However, it does not mention when not to use this tool or specify alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description provides key behavioral information: that existing labels are preserved. This adds value beyond the schema, but could note potential constraints (e.g., label uniqueness).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and key behavior, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple assignment tool, the description covers the essential behavioral nuance. Output schema existence is noted, but description doesn't mention return value, though not critical for agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and description does not detail parameters. However, param names ('device_id', 'label_id') and tool name make their purpose self-evident. Description could specify expected formats or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (assign), resource (label to device), and adds a crucial behavior (preserves existing labels), distinguishing it from sibling tools like labels_remove_label_from_device.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for adding a label without removal, but lacks explicit when-to-use or when-not-to-use guidance. Sibling context helps, but no direct alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states the action (dismiss all) but omits potential side effects, permission requirements, or scope details. For a simple action, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence. It front-loads the action and resource, with no unnecessary words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no annotations, output schema exists), the description is minimal but sufficient. It clearly states what the tool does. Could be improved by mentioning scope or confirmation, but it is complete enough for the use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100% (vacuously). According to guidelines, 0 parameters yields a baseline of 4. The description adds no parameter info, which is acceptable as there are none to describe.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Dismiss all persistent notifications.' uses a specific verb 'dismiss' and resource 'persistent notifications' with clear scope 'all'. It effectively distinguishes from the sibling 'services_notify_persistent_dismiss' which dismisses a single notification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates usage for bulk dismissal versus single dismissal via sibling tool name, but lacks explicit when-to-use or when-not-to-use guidance. No prerequisites are mentioned, but given the zero-parameter simplicity, the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description is clear that it's a read operation ('Get info'). No behavioral quirks expected; simple retrieval. Could mention it is read-only or safe, but not necessary given the simplicity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the verb and resource. Every word is informative. No wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read tool with an output schema, the description is sufficiently complete. It states the type of info returned. Could note that the output schema defines structure, but not necessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the description adds value by listing the information returned (version, arch, machine), which goes beyond the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'info about Home Assistant Core', specifying the exact fields (version, arch, machine). It distinguishes itself from other supervisor tools like 'supervisor_get_host_info' and 'supervisor_get_supervisor_info'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'supervisor_get_host_info' or 'supervisor_get_supervisor_info'. Usage is implied but not clarified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description accurately indicates a read-only retrieval operation with no side effects. However, it does not disclose potential prerequisites, error conditions, or authentication requirements beyond what is obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that conveys the essential purpose and available fields with no unnecessary words. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless retrieval tool with an output schema, the description lists the key fields (version, channel, healthy). It does not detail the output format, but the schema presumably covers that. The description is sufficiently complete for decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist in the input schema, so the description does not need to add parameter meaning. Baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves information about the Supervisor, specifying the fields: version, channel, healthy. It effectively distinguishes from sibling tools like supervisor_get_core_info or supervisor_get_host_info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or avoid this tool versus alternatives. Usage is implied by the description, but it lacks when-not scenarios or comparisons to similar get-info tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description alone must convey behavior. It states the tool lists active repair issues, which implies a read-only operation. It does not explicitly state side effects (e.g., no writes) or any authorization needs, but for a list operation, this is minimally adequate. A higher score would require explicit mention of read-only nature or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that directly state the purpose and provide examples. Every word adds value, with no redundancy or filler. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a straightforward list operation with no parameters and an output schema, the description provides adequate context. It explains what the issues are and why they need attention. However, it could be slightly more complete by mentioning that the result is a list of repair issues, but the output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema description coverage is 100%. The description adds no parameter-specific information, which is appropriate. A baseline score of 4 is given because the tool has no parameters; no additional description is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('List') and resource ('active Home Assistant repair issues'). It provides concrete examples of what constitutes a repair issue (deprecated config, broken integrations, required migrations). There are no sibling tools that list repair issues, so it is well-distinguished.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies the tool is for listing active repair issues that require attention, with examples. It implies the use case is to check for configuration or integration problems. However, it does not explicitly state when not to use this tool or mention alternatives, such as using system_get_updates for update-related issues.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It discloses merge behavior (merge vs replace) and the side effect of calling reload_themes unless reload=False. This covers key behavioral aspects, though it omits details like error handling or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: first states purpose, second explains merge, third explains reload. It is front-loaded, concise, and contains no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of sibling theme tools and an output schema (not detailed), the description covers essential behaviors. It explains the two configuration options but lacks details on error responses or prerequisites, which is acceptable for a simple update operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains the merge and reload parameters explicitly (e.g., merge=True merges, reload=False avoids reload), and implies name and variables as theme identifier and new variables. This adds significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'update' and resource 'existing theme file', distinguishing it from sibling tools like create, delete, or get. It also explains the key merge parameter behavior, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for modifying themes and details merge/reload options, but does not explicitly compare to other theme tools or provide when-not-to-use guidance. It lacks explicit context for choosing this tool over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It indicates a read-only operation ('Get'), which is accurate, but lacks details on error handling, permissions, or behavior when no preferred pipeline exists. Given simplicity, it's adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that conveys the core functionality with no extraneous words. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema (which documents return values), the description covers the primary retrieval modes. It could mention error cases or prerequisites, but for a straightforward get operation, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% coverage (no descriptions), so the description adds essential meaning: it explains that pipeline_id is optional and defaults to retrieving the preferred pipeline. This clarifies the parameter's role and default behavior beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single pipeline, either by ID or the preferred one if ID is omitted. This verb+resource combination is specific and distinguishes it from siblings like voice_list_pipelines or voice_create_pipeline.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly guides usage by specifying two retrieval modes, but does not explicitly state when to use this tool over alternatives like voice_list_pipelines. However, the context is clear enough for an agent to infer appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the key behavioral trait: the tool fetches current pipeline state to fill missing fields because HA's WS API requires the full set. This is a valuable transparency detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description uses two clear, efficient sentences with no fluff. Every word adds value: purpose and critical behavioral detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters and no annotations, the description explains the most important aspect (partial update). It is adequate for understanding core behavior, though additional context like error conditions or relationship to other voice tools could enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. While it explains the overall partial-update behavior, it does not describe individual parameters. The description adds some meaning beyond schema but not per-parameter detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing Assist pipeline,' which is a specific verb+resource. It distinguishes from siblings like voice_create_pipeline and voice_get_pipeline by implying update vs. create/get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool for updating pipelines and provides critical usage context about the partial update behavior ('Only fields you pass are sent'). It does not explicitly exclude alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds behavioral detail: accepts bare id or 'automation.<id>' format. Clarifies output is summaries/recent runs. No annotations provided, but this compensates somewhat.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded purpose then format detail. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description covers purpose and id format sufficiently. Could mention relationship to get_automation_trace for completeness, but adequate for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds critical meaning beyond schema: specifies accepted formats for automation_id (bare id or prefixed). Schema only says string, so this fully compensates for 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List', resource 'trace summaries (recent runs)', and scope 'for an automation'. Differentiates from siblings like automations_get_automation_trace by indicating it returns summaries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Implies usage for listing recent runs, but doesn't mention alternatives like automations_get_automation_trace for detailed traces.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not explicitly state that the tool is read-only or safe, but listing built-in templates is inherently non-destructive. The lack of explicit behavioral disclosure is a minor gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, front-loaded with the purpose, and contains no fluff. Every sentence adds value: the first states the action, the second details the output format, and the third explains the filter parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description does not need to detail return values, but it still provides the entry structure. With one optional parameter, the description fully covers what the tool does and how to use the filter, making it complete for its simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for the domain parameter, but the description compensates well by explaining that 'domain' filters to templates targeting a specific HA entity domain. It adds functional meaning beyond the schema's type and default value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists built-in card recipe templates, includes the structure of each entry, and explains the filtering option. This distinguishes it from sibling tools like card_builder_get_card_template or card_builder_list_cards.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the optional domain filter to narrow results, providing clear context for when to use it. It does not explicitly mention when not to use this tool or suggest alternatives, but for a simple listing tool, this is adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It implies a read-only operation returning a map, but does not explicitly state that it is non-destructive or requires no permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the essential purpose without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description adequately covers what the tool does without needing to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and schema coverage is 100%, so baseline is 4. The description adds no parameter info, which is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool provides a 'compact map of every Nexus tool namespace and its tool count,' which is specific and distinct from sibling tools like discover_tool_search and discover_get_tool_doc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context of being a compact overview implies use for getting a broad perspective before drilling down, but there is no explicit when-not-to-use or alternative mention.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry transparency burden. It indicates a read-only operation (get) and mentions the API source, but does not disclose auth requirements, rate limits, or other behavioral traits. Adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and target resource. No unnecessary words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and existence of output schema, the description fully captures the tool's purpose and context. No gaps identified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. Description adds no param-specific details, which is acceptable since there are no parameters to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool retrieves ESPHome add-on details (status, version, update availability) via the Supervisor API. Differentiates from sibling tools like esphome_get_config or esphome_get_addon_logs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies the tool returns addon info via Supervisor API, but does not explicitly contrast with supervisor_get_addon or provide when-not-to-use guidance. Context is clear but lacks exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly states what data is returned (YAML configs, registry entries, online status), indicating a read-only operation with no side effects. It does not disclose auth needs or rate limits, but for a simple listing tool this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys purpose and scope without extraneous information. Every phrase adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 params, has output schema), the description is complete. It specifies what is listed and with an output schema, return values are documented elsewhere.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema coverage is 100%. The description does not explain parameters because none exist. It implicitly covers the tool's scope well, meeting the baseline of 4 for no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists ESPHome devices and specifies three specific components: YAML configs, HA device registry entries, and online status. The verb 'List' and resource 'ESPHome devices' are specific and distinguish it from sibling tools like esphome_compile_device or devices_list_devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for ESPHome-specific listing but does not explicitly differentiate from alternatives like devices_list_devices or provide when-to-use/when-not-to-use guidance. It lacks explicit context for choosing this tool over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of disclosing behavioral traits. It states 'check reachability', which suggests a read-only operation, but does not explicitly confirm safety, permissions, or side effects. Basic transparency is present but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the tool's purpose with no extraneous words. It is perfectly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists (though not detailed here), the description adequately covers the essential information for a simple reachability check. It does not need to elaborate further.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the schema coverage is 100% by default. The description adds no parameter information, which is appropriate since none exist. The baseline score for zero parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Check' and the resource 'ESPHome dashboard reachability', specifying the URL constant. It effectively distinguishes this tool from all siblings, as no other tool performs a similar reachability check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing to verify dashboard reachability, but does not explicitly state when to use it over alternatives. Given there are no direct sibling tools for reachability checks, the guidance is clear enough for a simple operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states it lists branches (a read operation), but does not disclose any additional behavioral traits like performance, authentication needs, or scope limits (e.g., that it only lists local, not remote branches). Acceptable for a simple operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous information, fully front-loaded with the key action and scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is an output schema (presumably listing branches) and no parameters, the description is complete. It specifies the repository (HA config repo) and the type of branches (local), leaving no ambiguity about what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description does not need to add parameter semantics. According to the rubric, baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all local branches in the HA config repo, using a specific verb (list) and resource (branches in HA config repo). This distinguishes it from sibling tools like git_git_checkout_branch or git_git_create_branch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing local branches; since it's a simple list operation with no parameters, context is clear. No explicit when-not or alternatives are mentioned, but the sibling tools are distinct actions, so the purpose is sufficiently differentiated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description clearly indicates a read-only operation with no side effects. Since no annotations are provided, the description adequately discloses the behavior, though it could explicitly state that no modifications occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that conveys the essential information without any unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no parameters and an output schema, the description provides sufficient context: it specifies the resource and action completely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters in the input schema, so the baseline is 4. The description correctly has no parameter explanations, as none are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('show current git status') and the specific resource ('HA config directory'), distinguishing it from sibling git tools like commit, branch, or diff.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit usage guidelines are provided. The description implies the tool is for checking status, but it does not compare to other git tools or specify when to use it versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description indicates a non-destructive read operation. No side effects or behavioral traits beyond what is described. Annotations are absent, but the description fully covers the behavior without contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no redundancy. Efficiently conveys purpose and output fields. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, clear purpose), the description fully covers what the tool does and returns. Output schema exists but description complements it by listing fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Tool has zero parameters, so description does not need to add parameter meaning. Output fields are enumerated, providing value beyond schema. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'list' and resource 'installed add-ons' with specific fields returned (slug, name, state, version, update_available). Distinguishes from sibling tools like supervisor_get_addon which retrieves a single addon.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like supervisor_get_addon or supervisor_list_backups. However, for a simple list operation, the context is implied; still lacks explicit exclusions or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavior. It explains internal storage and default reload behavior, but lacks clarity on what happens if overwrite is false and the theme already exists (e.g., error or override).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with 4 sentences, front-loaded with the main action. Each sentence adds value without redundancy, and the structure guides the reader from purpose to details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 parameters, nested objects, output schema present), the description covers creation, parameters, and common usage patterns. It does not describe the output, but the presence of an output schema mitigates this. A brief note on failure cases would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description adds essential meaning: it explains 'variables' as CSS variables with examples, 'modes' for dark/light variants, and the purpose of 'overwrite' and 'reload'. This far exceeds what the schema alone provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create a new theme file') and resource ('themes/<name>.yaml' with CSS variables). It distinguishes itself from sibling theme tools (e.g., update, delete) through its focus on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use parameters like 'overwrite' and 'reload', and implies context for creating new themes. However, it does not explicitly mention when not to use this tool versus alternatives like 'update_theme' or possible error conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It clearly indicates a non-destructive list operation, which is straightforward. No additional behavioral traits (e.g., permissions, rate limits) are mentioned, but for a simple list it is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no filler. All information is front-loaded and every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with an output schema, the description provides all necessary information. It is complete and sufficient for an agent to understand the tool's purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has no parameters (0 params). According to rules, baseline is 4 when there are no params. The description adds no param info, which is acceptable since none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'todo.* entities', and the context 'currently registered in HA'. It is specific and distinguishes this from sibling tools like todo_list_items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives. The name implies it lists all todo entities, but there is no statement about when not to use it or comparisons with siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key behaviors: it accepts two forms of id and returns None if not found. Additional context like permissions or side effects is not needed for a read-only operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using two short sentences that cover purpose, input format, and return value. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the presence of an output schema, the description adequately covers input and return. It lacks error behavior or output structure, but the output schema fills that gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must explain the parameter. It does so by describing valid inputs (bare id or 'script.<id>'), adding meaning beyond the schema's type definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool gets a script's YAML config as a dict, specifies accepted input formats (bare id or 'script.<id>'), and mentions the return value when not found. This clearly distinguishes it from sibling tools like automations_get_automation_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving script configurations, but does not explicitly state when to use this vs alternatives. It provides context by specifying the input format and return behavior, which helps an agent decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the topics covered (hierarchy, state feedback, spacing, etc.) and implies it is a read-only reference. However, with no annotations provided, it could more clearly state that it returns information without side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short paragraphs: introduction, list of covered topics, and usage guidance. Every sentence adds value, no fluff. It is efficiently front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and having an output schema, the description explains what the tool provides comprehensively. It covers the scope and relationship to recipe_guide. It could mention read-only nature explicitly but is otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters and 100% schema description coverage, so the baseline is 4. The description does not need to add parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this tool is an 'Embedded UX design playbook for Card Builder cards' and distinguishes it from the sibling `recipe_guide()` tool by explaining that recipe_guide covers HOW to build a card while this covers WHAT to build for polished results. It lists specific design aspects covered.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Read this AFTER `recipe_guide()` when you're about to design a card that needs to look polished rather than just functional.' This provides clear ordering and context for use, though it could be more explicit about when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description effectively communicates that this is a read-only informational tool with no side effects. It explains what it covers (DocumentData layout, block tree, etc.) and its role in relation to build_from_recipe. This is sufficient for an agent to understand its behavior, though it could add details about being static or dynamic.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, front-loaded with the primary purpose. Every word adds value, and it efficiently conveys what the tool does and when to use it without any fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a guide with no parameters and a known output schema, the description covers the necessary topics and its relationship with sibling tools. It provides enough context for an agent to understand its role, though it could mention that it's a static reference or if it updates dynamically.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the schema coverage is 100% trivially. The description adds no parameter information, but none is needed. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's an embedded how-to guide for designing Card Builder cards programmatically. It distinguishes itself from sibling tools by being a reference guide rather than an action-oriented tool, and it explicitly lists the topics covered (DocumentData layout, block tree, etc.), making its purpose specific and distinguishable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Read this once at the start of a card-building task,' providing clear when-to-use guidance. It also mentions the recipe shorthand consumed by build_from_recipe, implying the guide is a prerequisite for that tool. However, it doesn't explicitly state when not to use this tool or list alternatives, though the context makes it reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description bears full responsibility for behavioral disclosure. It states it is a read operation and explains the parameter behavior, but does not elaborate on other aspects like idempotency or side effects. The description is adequate but not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using two sentences with no filler. The key information (purpose and parameter usage) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one optional parameter and an output schema, the description provides enough context. It covers the parameter variations and the default behavior, and the output schema handles return values. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description must compensate fully. It does so by explaining that url_path can be a dashboard name or a view path, with concrete examples. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'Lovelace dashboard config'. It distinguishes from sibling tools by specifying that it retrieves config, not a list or resources, and clarifies that it can return a dashboard or view config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool: to get config for a dashboard or a view. It provides examples of url_path values. However, it does not explicitly mention when not to use it or suggest alternatives, though the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description uses 'Force-rebuild' which indicates a destructive operation (overwrites the index), but does not disclose any potential side effects, impact on system performance, or whether it is safe to run frequently. With no annotations provided, more behavioral detail would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the action and its context. It is front-loaded with the verb 'Force-rebuild' and provides essential usage context, with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and an output schema (to explain return values), the description covers the core purpose and usage context. However, it could briefly mention if any preconditions or cautions apply (e.g., service interruption).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema has 100% coverage. The description does not add parameter information (as none exist), but according to the rubric, 0 parameters grants a baseline of 4. The description does not detract from this.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Force-rebuild the search index' with a specific verb and resource. It also provides context ('run after dynamically adding tools at runtime'), which distinguishes it from other discover tools like discover_tool_search or discover_list_namespaces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use the tool: after dynamically adding tools at runtime. This is a clear usage condition, and since no sibling tool performs a similar rebuild, there is no need to specify alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description covers key behaviors: grouping by domain, issuing one service call per domain, allowed actions, and return type. It does not mention error handling or side effects but is sufficiently transparent for a control tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states the core purpose, second adds grouping detail and return value. Every sentence is informative; no redundant text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple tool with output schema (return dict described) and no complex side effects, the description covers essential aspects. Minor omission: no mention of error handling or prerequisites, but overall complete for its class.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds critical value by enumerating allowed actions ('turn_on','turn_off','toggle') despite no schema enum, and explains entity_ids as a list of entities, clarifying usage beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it performs bulk turn_on/turn_off/toggle on entities, grouping by domain, which clearly distinguishes it from single-entity tools like entities_turn_on and entities_toggle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for bulk control of multiple entities and explains the domain grouping strategy, but does not explicitly state when not to use it or mention alternatives like entities_bulk_set_state.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses acceptance of two id formats and return of None on not found. No annotations provided, but description adequately covers behavior for a read operation. Could be more explicit about being read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words, front-loaded with purpose. Essential information efficiently conveyed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description covers parameter format, return behavior, and what config contains (entities + states). Complete for a simple get tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds value beyond schema by specifying accepted id formats (bare or prefixed). Schema coverage is 0% so description compensates well. Could include examples but sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'scene's config dict', and distinguishes it from sibling tools like automations_list_scenes and automations_set_scene_config. It also specifies accepted id format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit input format guidance ('bare id or scene.<id>') and return behavior (returns None if not found). Lacks when-not-to-use or explicit alternative selection, but implied sufficiently.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains the tool calls Home Assistant, checks entity and service registration, skips templates, and returns a detailed output structure. Lacks mention of permissions or side effects, but for a read-only validation tool, the behavior is well-disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences covering purpose, contrast, and limitation, followed by a clear return object definition. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no annotations and minimal schema, the description fully explains the tool's operation, inputs, outputs (listing all return fields), and its niche among many sibling automation tools. Complete for its complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one required string parameter yaml_content with 0% schema description coverage. The description mentions 'automation YAML' but does not elaborate on format, structure, or provide examples, leaving some ambiguity. Some value added beyond schema but could be more specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool cross-checks entity IDs and services from automation YAML against the live HA registry, distinguishing it from the static validate_best_practices sibling. The verb 'cross-check' and specific resources are well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly contrasts with validate_best_practices, stating this tool is dynamic and verifies existence, while the other is static. Also notes template values are skipped, providing clear guidance on when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility. It mentions side effects (initialization of a git repo) and the directory, but does not elaborate on idempotency, error conditions, or what happens if run multiple times. Adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with no unnecessary words. Every sentence adds value, making it very efficient for an AI agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, straightforward action), the description is fully complete. It covers the purpose, usage context, and prerequisite role. The presence of an output schema further reduces the need for additional explanation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema description coverage is 100%. The description adds no parameter information, which is appropriate as there are none to describe. Baseline 4 for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Initialize git repository'), the target resource ('in HA config directory'), and its role as a prerequisite ('Run once before using other git tools'). It effectively distinguishes itself from sibling git tools like commit, checkout, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use ('Run once before using other git tools'), implying it should not be used repeatedly or after other git operations. Provides clear context without needing additional explanation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses the return format, pagination, and filtering behavior. It does not explicitly state it is read-only, but the name 'snapshot_get' strongly implies no side effects. Additional mention of idempotency or safety would raise the score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with bullet points for each parameter, making it easy to scan. It front-loads the purpose. While every sentence adds value, it is slightly verbose; a more concise summary could improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 7 parameters, the description covers all aspects needed for correct invocation: filtering, pagination, output format. The existence of an output schema reduces the burden, but the description still provides crucial details. It is complete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 0%, but the description thoroughly explains all 7 parameters: include (valid values), domains, area_id, summary (effect), limit/offset (pagination), state_fields (valid keys). It adds significant meaning beyond the schema, including defaults and behavioral effects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides a 'one-shot aggregated view of HA state, filtered.' It names the resource (HA state snapshot) and action (get), and the sibling 'snapshot_get_area_snapshot' indicates area-specific vs general, so it distinguishes from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool by detailing its filtering capabilities and return structure. It implies it's for broad overviews, but lacks explicit 'when not to use' or direct comparison to siblings like 'snapshot_get_area_snapshot'. However, parameter guidance effectively communicates usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Warns about ~30s downtime and explains the safety prompt behavior. Without annotations, the description covers key behavioral traits adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, efficiently front-loading the action and warning, then explaining the parameter. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the essential: action, side effect, and usage. With an output schema present, the description is adequate, though it omits prerequisites or permissions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'confirm' is fully explained: setting it to true triggers restart, otherwise returns a safety prompt. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Restart Home Assistant' with a specific verb and resource. It distinguishes from siblings like system_stop_ha by focusing on restarting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use the tool (set confirm=True to proceed) and explains behavior without it (returns safety prompt). No exclusions or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description indicates a read-only operation ('List all... Returns groups...'), which is consistent with the tool name. No annotations are provided, but the description adequately communicates non-destructive behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, front-loading the core purpose and providing return details without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema (which exists but not shown), the description covers all necessary context: what it does, what it returns (members, icon, state). It is complete for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, the description adds essential meaning by specifying what is listed (entity groups) and what is returned (members, icon, state), fully compensating for the lack of schema detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all Home Assistant entity groups (group.* entities), specifying the exact resource type. It distinguishes itself from siblings like automations_list_automations by explicitly mentioning 'entity groups'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While no explicit when-to-use or alternatives are mentioned, the tool's purpose is self-explanatory as a list operation for groups, and siblings like automations_list_automations serve different entity types, making usage context clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that feature IDs are namespaced by domain and that passing domain filters results. It does not mention side effects (none expected) or performance limitations, but is transparent about filtering and naming convention.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and structured with bullet points, front-loading the purpose. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple interface (one optional parameter, output schema exists), the description is complete. It explains the key concepts (namespacing, auto value) without needing to cover return values since output schema handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage; the description adds full meaning for the single parameter 'domain'. It explains it filters by domain, gives examples of feature IDs, and explains the 'auto' value. This goes far beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists feature IDs for block-button-toggle/block-select-menu. It distinguishes from sibling card_builder tools by being specific to this block type, using a specific verb 'List' and resource 'feature IDs'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use: to get feature IDs for those blocks. It mentions filtering by domain and omitting to see all. It provides context like the 'auto' default. While no explicit when-not-to-use, the usage is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that it writes to /api/states (virtual state only), and provides return format. Lacks mention of idempotency or overwrite behavior, but given no annotations, it does well. Could add permission notes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four concise sentences: purpose, input format, return format, caveat. No filler. Front-loaded with key info. Every sentence is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers input format, return format, and the virtual vs real control distinction. Could mention partial failure handling, but the return format hints at it. Adequate for a simple bulk write tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema only defines an array of arbitrary objects. The description adds critical meaning: each item must contain entity_id and state, plus optional attributes dict. This compensates for 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it bulk-writes states using HA's set_state API. It specifies the input format (each item with entity_id, state, optional attributes) and return format. It also distinguishes itself from the sibling 'bulk_control' for real device control.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance: use this for virtual state writes, not for real device control. Directs to use 'bulk_control' instead for device control. No ambiguity about when to use this vs alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description warns about the need for manual restart and explains the safety prompt behavior. With no annotations provided, the description adequately discloses the destructive nature and safety mechanism.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with exactly two sentences, no wasted words. It front-loads the purpose and includes the warning and parameter guidance efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and an output schema, the description covers purpose, warning, and parameter behavior completely. No missing information is apparent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter 'confirm' has no schema description, but the tool description fully explains its effect: setting confirm=True proceeds with the stop, otherwise a safety prompt is returned. This compensates for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Stop Home Assistant', which is a specific verb and resource. It distinguishes from sibling tools like system_restart_ha by focusing on stopping, not restarting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions the need for a manual restart and the confirm parameter behavior. It provides clear context for when to use, but does not explicitly mention alternatives like system_restart_ha.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, but the description accurately describes the read-only operation and the specific path. It does not detail recursion depth or file extensions, but given simplicity, it is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema, the description fully explains purpose and use case, making it complete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (baseline 4); description adds no parameter info, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list YAML files') and the resource ('under themes/'), and distinguishes from sibling tools like themes_list_themes by specifying it lists files on disk, not loaded themes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides a usage scenario ('useful when a theme exists on disk but isn't loaded yet'), guiding the agent when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![ha-nexus-agent MCP server](https://glama.ai/mcp/servers/Fistacho/ha-nexus-agent/badges/card.svg)](https://glama.ai/mcp/servers/Fistacho/ha-nexus-agent)

Score Badge

Copy to your README.md:

[![ha-nexus-agent MCP server](https://glama.ai/mcp/servers/Fistacho/ha-nexus-agent/badges/score.svg)](https://glama.ai/mcp/servers/Fistacho/ha-nexus-agent)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Fistacho/ha-nexus-agent'

If you have feedback or need assistance with the MCP directory API, please join our Discord server