shelly-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.2
Disambiguation5/5
Each tool targets a distinct operation on a specific resource (e.g., switch, light, cover, KVS, scene, script). Actions like set, toggle, get, list, create, delete are separated per domain, and RPC read vs write are clearly divided. No two tools appear to do the same thing.
Naming Consistency5/5
All tools follow a consistent pattern: 'shelly_' + domain + verb (e.g., shelly_kvs_get, shelly_switch_set, shelly_scene_create). Verbs are uniform (set, get, list, create, update, delete, run, etc.) and the naming is predictable and clear.
Tool Count4/5
With 47 tools, the server covers a wide range of Shelly device operations (KVS, switches, lights, covers, energy, scenes, schedules, scripts, system, webhooks, etc.). While large, each tool serves a distinct purpose and the count reflects the complexity of managing Shelly devices. Slight reduction could be possible but overall appropriate.
Completeness5/5
The tool set provides full CRUD-like coverage for each subsystem: KVS (get, list, set, delete), switches/light/cover (set, toggle, move), scenes (list, get, create, run, delete), schedules (list, create, update, delete), scripts (list, get_code, create, put_code, start, stop, eval, delete), virtual components, webhooks, and system operations. No obvious gaps are apparent for typical Shelly management.
Average 3.9/5 across 47 of 47 tools scored. Lowest: 2.9/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 53 commits in the last 12 weeks
- Last stable release on June 10, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation idempotentHint: true already signals idempotency. The description adds that the action is 'Audit-logged', which is useful behavioral context beyond the annotation. However, it does not disclose potential side effects (e.g., triggering other actions) or response behavior, so it adds only modest value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short sentences, no redundant information. However, it is too brief for the tool's complexity. Front-loads the core purpose but sacrifices necessary details. Still, conciseness is high.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters with no schema descriptions, the description is grossly incomplete. It does not specify which fields can be updated, the nature of the 'calls' parameter, or the effect of optional parameters. An output schema exists but is not referenced. For a complex update operation, this level of description is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no parameter descriptions in the input schema). The description does not explain any of the 5 parameters (id, calls, device, enable, timespec) or their expected formats. For example, 'calls' is an array of objects with no further details, and 'timespec' is a string but not defined. The description adds no semantic value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing schedule by id, which distinguishes it from sibling tools like shelly_schedule_create, shelly_schedule_delete, and shelly_schedule_list. The verb 'Update' and resource 'schedule' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (create, delete, list). There is no mention of prerequisites, error conditions, or scenarios where the tool should not be used. The description is too brief to help an agent decide between it and sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the agent knows it is a safe read operation. The description adds that it uses Shelly.GetComponents and targets virtual (dynamic) components, which provides some additional context but does not reveal other behavioral traits like error handling or response structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, which is efficient for a simple tool. However, it lacks any structure (e.g., no examples or parameter details) that could improve readability without adding much length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema (not provided but stated in context) and only one parameter, the description is minimally adequate. However, it fails to explain the parameter or provide usage context, leaving gaps for an agent unfamiliar with the Shelly domain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'device' has no description in the schema (0% coverage). The description does not explain what 'device' refers to (e.g., device ID, name, or IP). It simply restates the method name, adding no clarity beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists virtual (dynamic) components via a specific method (Shelly.GetComponents). It distinguishes from sibling tools like shelly_list_components by focusing on virtual components, but could be slightly clearer on what constitutes a virtual component.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There are siblings like shelly_list_components, shelly_get_status, and shelly_get_config, but the description provides no context for selecting this tool over them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. The description adds that the response includes an etag, which is useful behavioral context. However, missing information on error behavior (e.g., if key not found).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, very concise. Front-loaded with the action. However, could be slightly expanded to include parameter context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema and annotations, the description still misses parameter details and usage context. For a simple get, it's partially complete but lacks explanation of inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for both parameters. Description only mentions 'key' but does not clarify its format or explain the 'device' parameter. No added meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single KVS value by key and includes its etag. This distinguishes it from sibling tools like 'shelly_kvs_list' (lists keys) and 'shelly_kvs_set' (writes).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. With many KVS siblings and similar read tools, explicit context would help avoid confusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears full burden. It mentions 'Audit-logged' as a behavioral trait, but lacks details on error handling, idempotency, or side effects of stopping a script.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and front-loaded with the action and return. However, it is too terse, omitting parameter explanations, which makes it less efficient for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 2 required parameters and an output schema, the description is minimal. It specifies the return value but lacks details about parameter semantics and usage context, making it barely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no meaning beyond parameter names. 'device' is not explained (e.g., IP or name), and 'id' is simply referenced without clarifying its role.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Stop a script by `id`' and the return value 'Returns `was_running`'. It distinguishes itself from sibling tools like shelly_script_start and shelly_script_create.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites (e.g., script must be running). The description only states what it does without comparative context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds valuable behavioral context beyond the destructiveHint annotation: it specifies that a confirmation parameter is required and that actions are audit-logged. This gives the agent important safety and logging awareness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (two sentences) and front-loaded with the core action. However, it may be too brief, omitting necessary parameter explanations, which trades conciseness for completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to document the three input parameters adequately. Given the tool's destructive nature and three parameters, the description lacks essential details like device identification, delay semantics, and confirmation behavior, making it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should clarify parameter meanings. It only addresses 'confirm' by noting it must be true, but leaves 'device' (required) and 'delay_ms' completely unexplained. No formats, defaults beyond schema, or constraints are provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Reboot a device' with a specific verb and resource. However, it does not differentiate from sibling tools like shelly_system_update, which also targets system-level actions, missing a chance to clarify distinct purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the requirement 'confirm:true', which serves as a usage prerequisite. However, no guidance is provided on when to use this tool versus alternatives (e.g., shelly_system_update for firmware updates), and no scenarios or exclusions are described.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description discloses credential field masking and the limitation for cloud-only devices, adding useful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences and front-loaded with the main purpose. It is reasonably concise, though the Gen1 detail adds length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a 'get config' tool with an output schema, the description adequately covers credential masking and error conditions but lacks parameter descriptions and does not explain what configurations are retrieved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description is expected to explain parameters. It provides no details about 'device' or 'component', failing to add meaning beyond the schema structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get a device's configuration' with specific verb and resource. It adds context about credential masking and cloud-only behavior, but does not explicitly differentiate from siblings like shelly_get_info or shelly_get_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'local-first' and that the tool returns an error for cloud-only devices, implying when not to use it, but does not name alternative tools or provide explicit when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description confirms read-only behavior, which is consistent with the readOnlyHint annotation. It adds value by specifying that the output includes 'ordered actions', which is beyond what annotations alone provide. However, it does not describe error handling or preconditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence + tag), but it omits critical information about the parameter. It is concise but under-specified, making it less helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to cover parameter semantics and usage guidance, leaving significant gaps for a tool that retrieves complex data. It is inadequate given the context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description does not explain the 'name' parameter (e.g., what it represents or its format). This leaves the agent with no guidance on how to populate the required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Show' and the resource 'a scene's full definition (its ordered actions)', which distinguishes it from sibling tools like shelly_scene_list (which lists scenes) and shelly_scene_create/delete/run (which modify state). The 'Read-only' tag reinforces its non-destructive nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives such as shelly_scene_list or shelly_scene_run. While it implies usage for retrieving a scene's details, it lacks contextual guidance on prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it returns 'was_running' and is audit-logged, providing some behavioral insight. However, with no annotations, it lacks detail on side effects, error handling, or behavior if the script is already running.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences. Every phrase adds value—no filler. The format is efficient and directly usable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of many sibling script tools and an output schema, the description is too sparse. It omits context about the device parameter, return structure, and prerequisites, leaving gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should add parameter meaning. It only mentions 'id' implicitly but does not explain the 'device' parameter or the format of 'id'. No value beyond the schema is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Start a script by id', which is a specific verb+resource pair. It effectively distinguishes from sibling tools like shelly_script_stop or shelly_script_eval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
While 'Start a script' is intuitive, there is no guidance on when to use this versus alternatives like shelly_script_eval or shelly_script_stop. The mention of 'Audit-logged' hints at tracking but doesn't clarify use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the read-only behavior is disclosed. The description adds 'Local-only', which provides additional context about scope. However, it does not disclose potential errors, rate limits, or authentication requirements, which would be valuable for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two short sentences) and front-loads the primary purpose. However, it sacrifices necessary detail, particularly about the parameter, which might require more space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple read-only list operation with one parameter and an output schema exists (not shown), the description provides the core purpose but lacks explanation of output format, pagination, or error scenarios. It is minimally adequate for simple usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 1 parameter ('device') with 0% coverage (no description in schema). The description mentions 'on a device' but does not clarify what format or identifier the 'device' parameter expects (e.g., ID, name, IP). This leaves ambiguity for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('webhooks configured on a device'). It also adds a scope qualifier ('Local-only'), which distinguishes it from potential cloud-based operations. The tool name and action are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings (e.g., shelly_webhook_create, shelly_webhook_delete) or any context about prerequisites. It does not mention alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds 'Audit-logged' which is beyond the idempotentHint annotation. However, it does not disclose other behavioral traits like permissions, idempotency implications, or failure behaviors. The addition is minimal but non-contradictory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences with no redundant information. It is front-loaded with the primary action and includes the critical behavioral note about audit logging.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 7 parameters (2 required) and siblings that create/delete/list, the description is too sparse. It omits how to use optional parameters effectively, what happens when updating non-existent webhooks, or the overall impact of updates. An output schema exists but is not described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should compensate by explaining parameter roles. It only mentions 'id' as the identifier, but does not clarify the optionality or semantics of parameters like 'name', 'urls', 'enable', etc. The schema structure is left entirely for the agent to infer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update fields') and the resource ('existing webhook'), and identifies a key differentiator ('Audit-logged'). It is specific and distinguishes from sibling tools like create, delete, and list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no conditions under which this operation is appropriate. It only mentions audit-logging, which is behavioral, not usage-oriented.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description contradicts the idempotentHint annotation by stating 'Create', which typically implies non-idempotent behavior. Additionally, it fails to add behavioral context beyond what the annotation provides.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with two sentences that front-load the purpose and provide essential next-step guidance without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers creation, return value, and next steps, it lacks explanation of prerequisites, idempotency behavior (due to contradiction), and other contextual details that would help the agent fully understand the tool's implications.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only mentions that the name parameter is optional, failing to explain the required device parameter. This does not sufficiently compensate for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates an empty script, optionally named, and returns its id. It distinguishes itself from siblings by mentioning the follow-up step with shelly_script_put_code.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions using shelly_script_put_code next and notes that the code step is confirm-gated, providing clear workflow context. However, it does not explicitly state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It discloses the toggle action (mutation) and audit logging, but lacks details on side effects, error conditions, or idempotency. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, front-loaded sentences with no unnecessary words. Every sentence adds value: purpose, return value, and logging trait.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool complexity (simple toggle, 2 params) and presence of an output schema, the description is adequate but minimal. It omits prerequisites (device existence), channel indexing, and potential constraints, but covers core behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, yet the description adds no parameter explanations. 'device' and 'channel' remain undefined; the description only loosely references 'switch/relay channel'. Does not compensate for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it toggles a switch/relay channel, specifies the return type as ChannelState, and notes audit logging. This verb+resource combination distinguishes it from siblings like shelly_switch_set (set vs toggle).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when wanting to flip a switch state but does not explicitly contrast with alternatives or provide when-not-to-use guidance. The audit-logged note adds context but not structural usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. Description adds that it reassembles paginated data, providing some behavioral context beyond annotations, but lacks details on response format or potential errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with main action, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with output schema present, so return values are covered. Missing info on edge cases (e.g., what if script not found) but adequate for basic usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage. Description only mentions 'device' in passing, does not clarify the 'id' parameter or device format. Fails to compensate for lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get a script's full source' which is a specific verb+resource, and adds detail about reassembling paginated GetCode, distinguishing it from siblings like shelly_script_put_code.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., shelly_script_list). No mention of prerequisites or context for its use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the description's mention of listing keys is consistent. It adds that etags are returned, which is useful behavioral context beyond annotations. However, it does not disclose other behaviors like pagination, permissions, or rate limits. Given the safety profile annotated, the description provides adequate but minimal additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence that immediately conveys the action and key detail (etags, wildcard default). Every part is essential, with no extraneous content. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only 2 parameters and an existing output schema, the description covers the tool's purpose and one parameter. However, it fails to mention the required 'device' parameter, which is a notable omission. For a list tool, the description is minimally adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds meaning for the 'match' parameter (wildcard pattern, default all), but completely omits the required 'device' parameter, which is critical for invocation. This partial coverage leaves a significant gap, so the description does not sufficiently augment the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'KVS keys', specifying that etags are included. It also mentions the wildcard pattern default, making the purpose unambiguous. This distinguishes it from sibling tools like shelly_kvs_get which retrieves a single key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing keys with optional filtering via 'match', but does not explicitly state when to use this versus alternatives like shelly_kvs_get or shelly_kvs_set. No exclusions or prerequisites are provided, leaving the agent to infer context from tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide idempotentHint=true. The description adds 'Audit-logged', which is a notable behavioral trait beyond what annotations offer. This helps an agent understand actions are tracked.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with verb 'Set'. No extraneous text. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Has output schema (not shown but noted), so return values are covered. However, no description of prerequisites or behavior like overwriting existing keys. For a simple set tool, it is moderately complete but missing some context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It clarifies 'value' is JSON but does not explain 'device' or 'key' parameters. Baseline for 0% coverage is low, and description adds only minimal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Set', resource 'KVS key', and target value type 'JSON'. This distinguishes it from sibling tools like shelly_kvs_get and shelly_kvs_delete, making purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when or when not to use this tool. The sibling list is large but no alternatives are mentioned. The audit-logged hint is the only contextual clue, but insufficient for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as destructive (destructiveHint: true). The description adds two critical behavioral details: the tool requires confirm:true to proceed and the operation is audit-logged. These extend beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise: one sentence for purpose, one for requirement, one for property. No wasted words, and information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While output schema exists, the description lacks context on prerequisites (e.g., script must exist), side effects (e.g., what if script is running), and clarification of the device parameter. Adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains that id identifies the script and requires confirm:true, but provides no meaning for device or the exact role of confirm. Partial compensation yields a middling score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states verb (delete), resource (script), and identifier (id). It distinguishes the tool from sibling script tools like shelly_script_start or shelly_script_get_code, as only this tool deletes scripts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., stopping a script instead of deleting it). The description only states what it does without contextual advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint:true, and the description adds value by noting that the result is stringified and operations are audit-logged. This provides additional context beyond the annotation, though it does not contradict it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences that convey the core purpose, a critical warning, and key behaviors. Every word serves a purpose with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description omits crucial details: the meaning of 'device' and 'id', the fact that the script must be already running, and the shape of the returned object. Significant gaps remain given the parameter count and schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must explain all parameters. It only clarifies 'code' (the expression) and 'confirm' (needed for arbitrary code), but leaves 'device' and 'id' completely unexplained. This is insufficient for proper usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Evaluate') and the resource ('expression inside a running script'), and implies the context of a Shelly device. It also highlights a key differentiator (arbitrary code needing confirm:true), which helps distinguish from other script tools like shelly_script_get_code or shelly_rpc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a specific guideline about using confirm:true for arbitrary code, but lacks explicit when-to-use vs alternatives, prerequisites (e.g., script must be running), or when-not-to-use. The guidance is present but incomplete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, and the description confirms a read operation with no side effects. The description adds value by disclosing the 'Local-only' constraint and the specific fields returned, which are beyond the annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise—one sentence with a parenthetical list—and front-loaded with the main purpose. Every part is relevant, but the lack of parameter explanation slightly detracts from overall efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown) and readOnlyHint annotation, the description adequately covers the tool's function and returned fields. However, the missing parameter semantics leaves a gap for a tool with a single required parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With schema description coverage at 0%, the description should explain the single parameter 'device', but it only implies it via the phrase 'on a device'. No format, constraints, or default values are provided, leaving the agent to guess.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'scripts on a device', and specifies the returned fields (id, name, enable, running). The 'Local-only' qualifier distinguishes it from remote operations, and the tool name differentiates it among sibling list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by naming the tool's action, but it lacks explicit guidance on when to use this tool versus alternatives (e.g., when to use shelly_script_list vs shelly_script_get_code). The 'Local-only' hint provides some context but no exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes validation of timespec, device limit, and method; audit logging. Adds behavioral context beyond idempotentHint annotation. Does not mention failure behavior or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no wasted words. Could improve readability with bullet points.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Provides validation context and logging, but does not explain return values (output schema exists). Lacks details on 'calls' structure beyond schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning to 'timespec' and 'calls' parameters not in schema, but does not describe 'device' or 'enable'. Schema coverage is 0%, so description partially compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a schedule' with specific resource and key parameters (timespec, calls). Distinguishes from sibling tools like shelly_schedule_update and shelly_schedule_delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for creation but lacks explicit guidance on when to use vs alternatives or when not to use. No statements about prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds behavioral context beyond annotations: it notes that the action is 'Audit-logged' and explains optional parameters (condition as JS expression, repeat_period semantics). Annotations only provide idempotentHint, so this adds value. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with clear separation of required and optional parameters. No fluff. Could be slightly more structured (e.g., grouping), but overall efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and no output schema provided, the description covers only a subset. Missing explanation for 'device' (required) and 'enable' limits completeness. Given the sibling webhook family, it provides adequate context for basic use but not enough for complex cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains event (with example), cid, urls (1-5), condition, and repeat_period. However, it omits several parameters including the required 'device' and optional 'name' and 'enable', leaving gaps in understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a webhook' and lists key parameters with examples (e.g., event 'switch.on'), making the tool's purpose unambiguous. It distinguishes itself from sibling webhook tools (list, update, delete) by its creation function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. The usage is implied by the tool name and sibling set, but there is no mention of prerequisites or scenarios where alternative tools (e.g., webhook_update) would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the readOnlyHint annotation by disclosing that the data is 'best-effort' and that detailed historical queries require local backends. This informs the agent about potential incompleteness and the tool's bounded reliability over cloud.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, concise and front-loaded. The first sentence covers purpose and a key limitation. The second adds technical detail but could be streamlined for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers output behavior and limitations, it fails to explain input parameters. The presence of an output schema helps, but the lack of input explanation leaves a significant gap in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% parameter description coverage in the schema, the description should explain the parameters but does not. It omits what 'device' and 'channel' represent or how to specify them, leaving the agent without necessary input context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it provides 'energy history' with 'lifetime totals' and 'per-minute series', distinguishing it from sibling tools like shelly_energy_live (live data) and shelly_get_status (current state). The verb 'retrieve' is implied and the resource is specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the tool: 'over cloud only totals + recent by-minute are available' and notes that 'richer history needs a local connection'. It does not explicitly name alternatives, but gives clear limitations that guide usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The destructiveHint annotation already marks the tool as destructive. The description adds behavioral details beyond that: the need for confirm:true and that the operation is audit-logged. These help the agent understand side effects without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short with two sentences and no unnecessary words. It is efficiently front-loaded with the action and constraint. Could be slightly longer to cover parameters, but the minimalism is acceptable for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 parameters, no nested objects, output schema exists), the description covers the core action, required confirmation, and audit logging. It does not explain the return value or error cases, but the output schema may cover that. Overall adequate but not fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It only explains the confirm parameter (must be true), leaving key and device undescribed. While these may be self-explanatory, the description does not add meaning beyond the schema's field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a KVS key', specifying the verb (delete) and the resource (KVS key). This distinguishes it from sibling tools like shelly_kvs_get, shelly_kvs_set, and shelly_kvs_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes the requirement for confirm:true, which guides the agent on how to invoke the tool. It also mentions audit-logged, implying usage context. However, it does not explicitly state when not to use or compare to alternatives, but the core guidance is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true. The description adds value by mentioning the return of 'ChannelState' and that the action is audit-logged, providing behavioral context beyond structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action, followed by optional behavior, return value, and audit notice. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description sufficiently covers return and audit. However, it could provide more context on required parameters like device identification, but overall it's adequate for a switch control tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Parameter schema coverage is 0%, but the description only hints at the 'toggle_after_s' parameter via 'optionally auto-revert after N seconds'. It does not explain 'device', 'channel', or 'on' parameters, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Turn a switch/relay channel on or off' with a specific verb and resource. It also highlights the optional auto-revert feature, distinguishing it from sibling tools like shelly_switch_toggle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for on/off control but does not explicitly state when to use this tool versus alternatives like shelly_switch_toggle. It lacks explicit when-not or contextual exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, and the description confirms 'Safe, read-only.' No additional behavioral traits (e.g., rate limits, authentication requirements, side effects) are disclosed. With annotations covering the safety profile, the description adds no new behavioral insight beyond confirming the read-only nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences with no wasted words. The core purpose is front-loaded in the first sentence, specifying the information retrieved. The second sentence clarifies the parameter. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has a single required parameter and an output schema (present), the description adequately lists the returned device attributes: generation, model, firmware, online state, and capabilities. It does not mention any constraints (e.g., only one device per call) or edge cases, but for a simple retrieval tool this is sufficient. The output schema can provide the full return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the sole parameter 'device' (type string). The description compensates by explaining that 'device is a device id or a configured name,' adding essential semantic meaning. This clarifies what string values are acceptable, significantly aiding correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool identifies one device including generation, model, firmware, online state, and capabilities. It specifies the parameter 'device' as a device id or configured name. This distinguishes it from sibling tools like shelly_get_status (status) and shelly_get_config (configuration), providing a specific verb and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description notes it is 'Safe, read-only,' implying low risk and appropriate for any context where device info is needed. However, it does not explicitly state when to use this tool versus alternatives (e.g., shelly_get_status for live state, shelly_list_devices for all devices), nor does it provide when-not-to-use guidance. More explicit usage direction would be beneficial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. The description adds 'Safe, read-only,' which reiterates the annotation but offers no further behavioral details (e.g., error handling, permissions). Minimal added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with an example. No redundant text. Information is front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description adequately covers purpose and usage guidance, but fails to clarify the format of the device parameter. Since an output schema exists, return values are not needed, but parameter ambiguity is a gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one required string parameter 'device' with 0% schema description coverage. The description mentions 'a specific device' but does not specify format (e.g., IP address, device ID), leaving ambiguity for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists component keys on a device with an example. It distinguishes from siblings like shelly_list_methods, which lists RPC methods.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this to discover what a specific device exposes before reading status or controlling it,' providing clear when-to-use context and implying alternatives (read/control tools).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond the destructiveHint annotation by detailing chunking, append behavior, and audit logging. It does not contradict annotations and provides useful context about the confirm flag and arbitrary code execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at three sentences, uses bold for emphasis, and has a clear structure. Every sentence adds value without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description appropriately focuses on input behavior. It covers critical aspects like chunking, append modes, and the confirm safety flag. Minor gaps include lack of mention of script existence prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description explains the 'append' and 'confirm' parameters and their implications. However, it omits explanation for 'id' and 'device' parameters, which are required and could be clarified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (upload code), the resource (script), and mentions chunking and the confirm flag. It distinguishes from siblings like shelly_script_get_code and shelly_script_create.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains chunking and append behavior but does not explicitly state when to use this tool vs alternatives such as shelly_script_eval or shelly_script_create. Usage context is implied but not explicitly defined.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide destructiveHint:true, so the destructive nature is clear. The description adds value by requiring 'confirm:true' for the operation, which is a behavioral trait beyond the annotation. No contradiction detected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the main purpose. Every part earns its place with no superfluous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential aspects: action, resource, key format, and the confirm requirement. Given that an output schema exists and annotations mark it destructive, the description is largely complete for a delete operation, though the missing device parameter info is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate. It explains the key parameter format ('<type>:<cid>') and implies the confirm parameter must be true, but the device parameter is not described at all. This provides partial but incomplete parameter guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a virtual component by key' with the key format specified as '<type>:<cid>'. It directly conveys the action and resource, distinguishing it from sibling tools like shelly_virtual_list (list) and shelly_virtual_add (add).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions 'Requires confirm:true', giving a clear prerequisite for deletion. However, it does not provide guidance on when to use this tool versus other delete tools (e.g., shelly_kvs_delete, shelly_scene_delete) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only provide destructiveHint:true. The description adds two behavioral traits beyond annotations: the requirement for confirm:true (a safety guard) and that the operation is audit-logged. This provides useful context for an AI agent deciding to invoke the tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence with three clear clauses. It is front-loaded with the primary action, followed by two critical usage notes. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (simple delete with 3 params) and the existence of an output schema, the description covers the core action and key behavioral aspects (confirm, audit). However, it lacks details about the 'device' parameter and the response structure, which the output schema may address, but the description does not fully ensure the agent can call the tool correctly without guessing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the 'id' parameter (delete by id) and the 'confirm' requirement, but does not clarify the 'device' parameter. With three undocumented parameters, this is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete a webhook' by id. The verb 'delete' and resource 'webhook' are specific, and the sibling tools (e.g., shelly_webhook_list, shelly_webhook_create) are distinct, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes the requirement 'confirm:true', indicating that deletion only occurs with explicit confirmation, which helps the agent avoid accidental destructive actions. It does not explicitly contrast with sibling tools, but the name and context imply 'use this to delete a webhook, not to list or create'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only. The description adds that certain devices return None for unsupported fields (e.g., Gen1 plugs lack voltage/current), providing useful behavioral context beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences plus a note, no wasted words, and the core purpose is front-loaded. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description covers the key fields and edge cases (None values for unsupported devices). However, it does not explain that the 'device' parameter specifies which Shelly device to query, which could be assumed but is not explicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description only mentions the 'channel' parameter ('Pass channel to narrow to one') but does not describe the required 'device' parameter, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves live power/energy per channel and lists the specific fields (power_w, voltage, etc.). The verb 'Live' and resource 'power/energy per channel' precisely define the scope, distinguishing it from related tools like shelly_energy_history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to narrow to a single channel using the 'channel' parameter and notes that it is safe and read-only. However, it does not explicitly contrast with sibling tools or specify when not to use it, leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond annotations (readOnlyHint) by specifying that it returns an actionable error on Gen1/cloud. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundancy. First sentence states purpose, second adds important error behavior. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool with one parameter and an output schema, the description covers main aspects but misses parameter format guidance. Could briefly hint at device identifier format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not elaborate on the 'device' parameter beyond its name. Schema coverage is 0%, so the parameter lacks format or value constraints in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists RPC methods for a device, and distinguishes between Gen2+ and Gen1/cloud, making its purpose specific and distinct from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context by mentioning Gen2+ support and actionable error on Gen1/cloud, but does not explicitly state when to use or exclude alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint: true, and description adds that confirm is required for deletion, which is a key behavioral trait. This adds value beyond the annotation without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no redundant information. Every word earns its place, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete operation with only two parameters, the description covers the essential action and confirmation requirement. However, it omits behavior on non-existing scenes or error states, and there is no output schema provided in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains that 'confirm' is required for deletion but does not elaborate on the 'name' parameter aside from implying it identifies a scene. Lacks details on existence or uniqueness constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete a named scene', which matches the tool name and distinguishes it from sibling tools like create, get, list. It specifies the action and resource precisely.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explicitly mentions the requirement for confirm:true, guiding the agent on how to invoke the tool safely. However, it does not provide when-not-to-use or alternatives, though for a delete operation this is acceptable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. Description adds 'Local-only (cloud can't manage schedules)' as a behavioral constraint. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action and resource. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with output schema and annotations, description covers purpose and constraint. Could mention return format, but output schema handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one parameter 'device' with no description. Coverage is 0%. Description does not elaborate on what 'device' expects (e.g., ID, name). Needs more detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'list the schedules configured on a device', specifying verb and resource. The local-only qualifier distinguishes it from cloud operations. Sibling tools confirm it's a read-only list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a use case: use locally, not in cloud. Implicitly tells when not to use (cloud). No explicit alternatives, but adequate for a simple list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool moves a cover, returns a CoverState, and is audit-logged. It does not mention error behavior, idempotency, or permissions, but the core behavior is well-communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose, and contains no irrelevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and presence of an output schema, the description covers most necessary context: action/position modes, return type, and audit logging. Missing explanation of device or channel is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for action (open|close|stop) and position (0-100) but does not explain device or channel. With 0% schema coverage, this is partially helpful but incomplete for all parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool moves a roller/cover with action (open/close/stop) or position (0-100). It is distinct from sibling tools, which focus on switches, lights, scripts, etc., making its purpose very clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies how to use the tool (choose action or position) but does not explicitly state when not to use it or what scenarios are unsupported. However, given no alternative cover tool, the implied usage is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds 'Audit-logged' beyond the destructiveHint annotation, providing extra behavioral context. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. Essential information is front-loaded: what it does, the requirement, and an extra behavioral note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with output schema and destructiveHint annotation, the description covers purpose, a key requirement, and audit-logging. It is sufficient but could mention error handling or confirmation necessity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the `id` parameter as the schedule identifier and explicitly mandates `confirm:true`. However, the `device` parameter is not described, and with 0% schema coverage, more detail would be beneficial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the resource ('a schedule'), and the identifier ('by `id`'). This distinguishes it from sibling tools like shelly_schedule_create, shelly_schedule_update, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly requires `confirm:true`, which is a crucial usage precondition. While it doesn't mention alternatives, the requirement is clearly stated, aiding correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true. The description adds valuable context: password must be ≥12 chars, is write-only, never returned, and redacted in audit log. This goes beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the action and critical constraint. Every sentence adds value with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple tool with an output schema (not shown) and annotations, the description covers key behavioral traits and constraints. It lacks detail on the device parameter but is otherwise complete for selecting and invoking the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It specifies that password must be ≥12 chars and that confirm must be true, adding meaning beyond schema field names. However, the device parameter is not described, which is a gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb+resource: 'Enable/rotate the device login password'. It clearly distinguishes from sibling tools like shelly_system_reboot or shelly_system_update, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the main use case (enable/rotate password) and a key constraint ('Requires confirm:true'). It does not explicitly mention when not to use or provide alternatives, but the context is clear for a password management tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint: true. The description adds value by specifying the channel parameter values (stable|beta) and the confirmation requirement, which are behavioral constraints beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It front-loads the action and immediately follows with critical parameter constraints. Every part is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the destructive nature and existence of output schema, the description provides essential information (action, channel options, confirmation). It omits details like prerequisites or side effects but is adequate for a straightforward firmware update tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates by explaining the channel parameter's allowed values and clarifying that confirm must be true. However, the device parameter remains unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Trigger a firmware update' with the verb 'trigger' and resource 'firmware update'. It distinguishes from sibling tools like shelly_system_reboot by specifying a different action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a usage condition ('Requires confirm:true') but does not explicitly guide when to use this tool versus alternatives. The name implies firmware updates, but no when-not-to-use or comparative guidance is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide idempotentHint: true, and the description adds that the operation is audit-logged. This adds context beyond annotations, though the idempotence implication of 'Add' is not fully clarified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences front-load the action and type list, then cover optional parameters and return value. Every sentence adds value with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists and the description mentions the return value, it covers the core lifecycle. Minor gaps: no mention of prerequisites (e.g., device existence) or error conditions, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description does some work: it explains id's range (200-299) and that config is optional. But it does not describe the 'device' or 'type' parameters beyond naming them, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a virtual component, specifies the allowed types (boolean/number/text/enum/button/group), and distinguishes from sibling tools like shelly_virtual_list (list) and shelly_virtual_delete (delete).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for creating virtual components but does not explicitly contrast with alternatives or state when not to use. However, the context is clear enough for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond the idempotentHint annotation: it notes that only provided fields are changed (incremental update), is audit-logged, and returns the post-action LightState. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (few sentences) and front-loaded: it begins with the core purpose and then breaks down parameters. However, it mixes range explanations and notes, and could be more structured with bullet points or clearer sections.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (8 parameters, optional fields) and the presence of an output schema (not shown but noted), the description covers the main behavior and return value. It lacks detailed parameter explanations and does not explain the channel parameter or transition_s, which could lead to incomplete agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description partially compensates by explaining brightness (0-100), rgb (three 0-255 values), temp_k (white temperature). However, it omits details for on, white, channel, transition_s, and the exact format of rgb array. This leaves gaps for an agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets a light/dimmer/RGB(W)/CCT channel, specifying it works for both Gen1 and Gen2 devices. It distinguishes from sibling tools like shelly_switch_set (switches) and shelly_cover_move (covers), so the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by listing updatable fields and giving parameter ranges (brightness 0-100, rgb three values, temp_k). It does not explicitly state when to avoid this tool or mention alternatives, but sibling tools cover different domains (switches, covers, scripts, etc.), making the context clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint and openWorldHint. The description adds that mutating methods are refused and unknown methods are treated as writes and rejected, which provides behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no redundant words. The key constraint (read-only, alternative tool) is front-loaded. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Has output schema, so return values need not be explained. The description covers purpose, usage constraints, and behavioral traits. However, it lacks details on parameter syntax or format, which is not fully compensated by the schema. Still, for a generic RPC proxy tool, it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should compensate but does not explain the meanings of 'device', 'method', or 'params' parameters individually. It only gives method naming patterns, leaving the agent to guess the exact inputs. High-level context is present but insufficient for parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool calls read-only RPC methods on a device, specifying the method patterns (*.Get*, *.List*, *.Check*). It explicitly distinguishes from the sibling tool shelly_rpc_write.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use (read-only methods) and when-not-to-use (mutating methods) with direct reference to the alternative tool shelly_rpc_write. Also notes that unknown methods are rejected server-side.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide idempotentHint=true; the description adds requirements for methods and devices, failure conditions, and a warning about Toggle methods. This adds significant context beyond the annotation, revealing safe usage patterns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences) and front-loads the purpose. Every sentence adds value, but could be better structured (e.g., separate constraints from advice). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 params, 2 required) and presence of an output schema, the description covers the essential behavioral aspects: action structure, method constraints, overwrite behavior. It does not mention return values (covered by output schema) or rate limits. Minor gap for 'description' parameter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description compensates by explaining the structure of 'actions' as ordered list of device/method/params and the meaning of 'overwrite'. It does not describe 'name' (obvious) or 'description' (minor gap). Overall adds substantial meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool defines/saves a named scene from an ordered list of actions, which is a specific verb+resource. It distinguishes itself from sibling scene tools (run, delete, get, list) by focusing on creation. The structure of actions is also explained.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit constraints: methods must be non-destructive mutating, devices must be known, overwrite behavior for existing names, and a preference for Set methods over Toggle. It gives clear context but does not explicitly compare to alternative scene creation tools (none exist) or state when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds 'Read-only' which reinforces the existing readOnlyHint annotation. It does not provide additional behavioral details beyond what annotations already convey, such as any side effects or access requirements, but is consistent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. It front-loads the primary purpose ('List defined scenes') and adds only relevant detail about returned fields. No extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, an output schema exists, and annotations provide safety information, the description is complete enough. It specifies the output fields, making it sufficient for an agent to understand the tool's capability.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100% (vacuously). With zero parameters, the baseline is 4, and the description adds no parameter information because none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists defined scenes and specifies the included fields (name, description, action count). This distinguishes it from sibling scene tools like shelly_scene_get, shelly_scene_create, shelly_scene_run, and shelly_scene_delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates a listing operation with 'List defined scenes' and emphasizes it is read-only. While it does not explicitly mention when to avoid this tool, the context and sibling names make the usage clear for a simple list operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description covers the key behavioral aspect: it returns version info and is a health check. It implies read-only, no side effects, which is sufficient for such a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no fluff, front-loaded with the core action and purpose. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and an output schema, the description is complete: it identifies the return value (version) and use case (health check). No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the description does not need to add param info. Baseline 4 applies as per guidelines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns the server version and acts as a health check, which is specific and distinct from all sibling tools that focus on device operations, settings, or other queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool or provide alternatives, but its purpose is simple and self-evident as a health check, making guidance less critical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=true. The description adds context by stating 'Safe, read-only' and 'returns lightweight identities (probe for detail),' but does not disclose additional behavioral traits beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences plus parameter explanations, front-loading the purpose with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and low complexity, the description adequately covers the tool's purpose and usage without needing to detail return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description explains both parameters: timeout_s as the mDNS browse window and use_cloud to skip the cloud list, adding significant meaning not present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Discover Shelly devices on the LAN via mDNS, merged with the cloud account list,' which provides a specific verb (discover) and resource (Shelly devices). It distinguishes from sibling tools like shelly_list_devices by focusing on network discovery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to use parameters (timeout_s for browse window, use_cloud to skip cloud list) and notes safety and output nature. However, it does not explicitly state when not to use the tool or compare with alternatives like shelly_list_devices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond the readOnlyHint annotation by explaining the normalization across Gen1/Gen2/Cloud and the return structure (canonical channels plus raw payload). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise: two short paragraphs with clear front-loading of the main purpose. Every sentence adds value, no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the output schema likely documents return values, the description provides a high-level summary of what is returned. The two parameters are adequately described, and the tool's normalization and safety are covered. It is complete for this tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description compensates by explaining the 'component' parameter with an example. While 'device' is not elaborated, it is a common required parameter and self-explanatory. The description adds significant value over the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get a device's **normalized** live status'. It specifies the resource (device status), the verb (Get), and the unique normalization aspect that distinguishes it from other getters like shelly_get_info or shelly_get_config.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains when to use the optional 'component' parameter: 'Pass ``component`` (e.g. ``"switch:0"``) to narrow to a single component.' It also notes the tool is safe and read-only. While it doesn't explicitly list alternatives, the sibling context implies this tool is for live status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark readOnlyHint: true. The description adds that it is 'Safe, read-only' and specifies the returned fields (id, name, generation, model, online state), providing behavioral detail beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence states the core function, the second adds safety and return value context. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 0 parameters and an output schema present, the description adequately explains the tool's purpose, safety, and return format. It is complete for a list tool given the context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters, so no parameter descriptions are needed. The baseline for 0 parameters is 4, and the description does not need to add parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'every Shelly device known to this server', and specifies the scope 'configured + cloud account'. It is distinct from sibling tools like shelly_discover or shelly_get_info by focusing on listing all known devices.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context by calling it 'the starting point before any status or control call', which guides the agent to use it early in a workflow. It does not explicitly mention when not to use it or suggest alternatives, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint, openWorldHint), the description adds that mutating calls require confirm:true, are audit-logged, and destructive methods need params.i_understand_data_loss = true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with key information, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description covers mutation types, confirmation requirement, and destructive safety, though it might briefly note the params object is flexible.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description adds essential context for the confirm parameter and the destructive gate parameter, though it does not explicitly define each parameter individually.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it calls any mutating RPC method, using the verb 'call' and specifying 'mutating' to distinguish from the read-only sibling shelly_rpc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says to use shelly_rpc for read methods and describes the extra gate for destructive methods, providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses best-effort execution, sequential action attempt, non-aborting on failure, possible statuses ('ok', 'partial', 'failed'), re-run safety due to absolute states, and audit logging. This is comprehensive behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently front-loaded with the core action. It uses 6 sentences, each adding value (behavior, status codes, re-running, audit logging). Slightly verbose but still clear and without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter) and the presence of an output schema, the description covers behavioral expectations, error handling, and idempotency. No additional information is necessary for an agent to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is one parameter 'name' with no schema description. The description compensates by referring to 'named scene', indicating that the parameter identifies which scene to run. Although it doesn't specify constraints (e.g., valid values), the simple nature of the parameter and context make it clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a named scene, attempting each action in order, and reporting per-action results. It uses a specific verb 'Run' and identifies the resource as a 'named scene'. It distinguishes from sibling tools like shelly_scene_list and shelly_scene_get which list or retrieve scenes, not execute them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context that the tool is for executing scenes, and explains the best-effort, sequential behavior. It implies the correct use case (running a scene) but does not explicitly exclude alternatives. However, given sibling tool names, the purpose is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![shelly-mcp MCP server](https://glama.ai/mcp/servers/Buggy1111/shelly-mcp/badges/card.svg)](https://glama.ai/mcp/servers/Buggy1111/shelly-mcp)

Score Badge

Copy to your README.md:

[![shelly-mcp MCP server](https://glama.ai/mcp/servers/Buggy1111/shelly-mcp/badges/score.svg)](https://glama.ai/mcp/servers/Buggy1111/shelly-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Buggy1111/shelly-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server