Server Quality Checklist

Tool Scores

• deep_linkC
  2.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description says 'Launch an app' which implies a write/mutation operation, but the annotation readOnlyHint=true indicates this is a read-only operation. This is a clear contradiction. The description does not disclose any other behavioral traits beyond what annotations already provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, directly states the purpose, and provides an equivalent API call with no wasted words. It is well-structured and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 4 parameters, 50% schema description coverage, and no output schema, the description is too brief. It does not cover parameter details for appId and device, nor does it describe the return state or side effects. The contradiction with annotations further reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning for contentId and mediaType, but the required appId parameter and the device parameter are not explained. With schema description coverage at 50%, the description partially compensates but leaves gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool launches an app with a deep link using contentId and mediaType. It provides an equivalent ECP endpoint, making the purpose specific. However, it does not explicitly differentiate from the sibling tool launch_app, which might serve a similar function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for deep linking via the mention of contentId and mediaType and the ECP equivalent. However, it lacks explicit guidance on when to use this tool versus alternatives like launch_app, and does not state prerequisites or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• input_textB
  3.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description indicates a mutation (sending text), but the annotation 'readOnlyHint' is set to true, which contradicts the described behavior. The description does not address this inconsistency or provide any behavioral details beyond the contradictory implication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence of 14 words with no unnecessary information. It efficiently communicates the tool's core action and target.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite good purpose clarity and conciseness, the description lacks usage guidance, does not resolve the annotation contradiction, and omits important behavioral context. For a tool with two parameters and no output schema, the description is incomplete for effective agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning the schema already describes both parameters adequately. The tool description does not add any additional meaning beyond what the schema provides, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Send' and the resource 'literal text string' to a specific target 'whatever input field is currently focused on the device', which unambiguously identifies the tool's function. It also references the ECP endpoint, adding context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for sending text input to a focused field, but it does not provide any guidance on when to use it versus alternatives like 'keypress' or when not to use it. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keypressA
  3.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states it sends a remote key, which is a write/mutation action, but annotations declare readOnlyHint=true and idempotentHint=true, which contradicts the described behavior. The description does not correct or address this discrepancy.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundant information. The first sentence clearly states the action, and the second provides a helpful analogy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters, no output schema, and exhaustive schema coverage, the description is complete. It explains what the tool does, gives examples, and provides an analogy for context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description provides examples for the 'key' parameter and references another tool for the full list, but this adds minimal value beyond the schema. For the 'device' parameter, the description repeats the schema's description exactly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Send' and the resource 'ECP remote key' to a Roku device, with concrete examples like 'Home', 'Up', 'Select', 'Play'. It also distinguishes itself from siblings (e.g., input_text, deep_link) by specifically targeting remote key presses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says it 'mirrors what the user does with the on-screen remote,' which implies its use case but does not explicitly state when to use it versus alternatives like input_text or deep_link. No exclusions or when-not conditions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rale_get_node_by_idA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint. The description adds that the tool is 'Read-only' and mentions the default for path, but does not provide significant behavioral context beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with two short sentences. It front-loads the key information (read-only, function name, required and optional parameters) without any extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with no output schema and good annotations, the description adequately covers purpose and parameters. It could mention the expected return format, but the name implies a node object is returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, baseline is 3. The description adds the default value for the path parameter (default []), which is not explicit in the schema. This provides additional clarity beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool is a read-only operation via App Connector, with specific verb and resource ('getNodeById'). However, it does not explicitly differentiate from sibling tools like 'rale_command'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, constraints, or scenarios where this tool should or should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_app_iconA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false. The description adds only the return format (data URL/base64), which is useful but does not disclose new behavioral traits beyond what annotations provide. It is consistent with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no wasted words. It is front-loaded and efficiently conveys the core action, resource, dimensions, and output format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with few parameters and no output schema. The description covers the basic purpose but omits context like what a 'channel' is or whether the icon is device-specific. It is minimally adequate but could benefit from clarifying the meaning of appId and the nature of the icon.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%: only 'device' has a description. 'appId' lacks any schema description, and the description does not clarify its meaning or usage. The description hints at 'channel' but does not fully explain parameters, failing to compensate for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Fetch', the resource '336x210 app icon' for a channel on a device, and the return format (data URL/base64). This unambiguously identifies the tool's function and distinguishes it from sibling tools like 'launch_app' or 'sideload'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching an app icon but provides no explicit guidance on when to use this tool versus alternatives, nor any conditions or exclusions. The tool's purpose is clear, but the description lacks when-to-use or when-not-to-use context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_selected_deviceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, and idempotentHint, indicating safe, non-mutating, idempotent behavior. The description adds that it returns the 'currently focused' device, implying statefulness, which is useful context. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 10 words, directly states purpose with no extraneous information. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with no parameters and no output schema, the description is adequate but could optionally mention what the returned device information contains (e.g., device ID or name). However, given the tool's simplicity and the presence of annotations, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters with 100% coverage. The description adds meaning by clarifying that no input is needed, and that the tool returns the device the user is currently focused on. Since there are no parameters, baseline is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool retrieves the device the user is currently focused on in Dev Studio. The verb 'get' is implied, and 'selected device' is specifically defined as the current focus, distinguishing it from sibling tools that retrieve different types of information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, nor any conditions for when not to use it. Since it has no parameters, it's a straightforward read, but the description does not provide any context about when it is appropriate to call.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_action_schemaA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description does not need to repeat these. It adds clarity about the required argument but does not disclose additional behavioral traits beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The description is front-loaded with the core purpose and immediately specifies the required parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single enum parameter and no output schema, the description adequately covers purpose and parameter constraint. It is complete enough given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description restates the enum constraint already present in the schema. It adds no new meaning beyond what the schema provides, so baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the schema for one Action step type, using specific verb 'Return' and resource 'schema'. It distinguishes from sibling tools which are actual actions (e.g., keypress, launch_app) rather than schema queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the required argument 'type' and that it must come from list_action_types (enumerated in inputSchema). It implicitly guides when to use (when you need a schema for a step type), but does not explicitly mention when not to use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch_appA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, openWorldHint, idempotentHint, destructiveHint. Description adds only minor context ('dev' is sideloaded), not substantial behavioral disclosure beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. Front-loaded with main action. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but launch action likely has minimal return. Parameters are well-covered. Annotations rich. Sufficiently complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. Description adds meaning by explaining special appId values ('dev'), device format (IP or serial), and that params are URL-encoded, adding value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it launches a channel/app by app id. Provides specific verb and resource, and distinguishes from siblings like sideload (install) and keypress (remote).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells agents to use /query/apps or ecp_query to discover app ids. Does not specify when not to use, but usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_sideloadA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true and readOnlyHint=false, so the tool's mutation nature is clear. The description adds context about the password being optional based on remembered credentials, which goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first states the core action, and the second clarifies a parameter usage. Every word is necessary and the structure is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (2 optional parameters, no output schema), the description adequately conveys what the tool does and how to use it. The destructive nature is covered by annotations, and the password handling is explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description reinforces the password optionality but does not add significant new meaning beyond what the schema already provides for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Remove the currently sideloaded Dev App from the device.' It uses a specific verb 'Remove' and identifies the exact resource. This distinguishes it from the sibling tool 'sideload' which likely uploads rather than deletes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage context: removing a sideloaded app and clarifying when to omit the password ('when Dev Studio has remembered it'). However, it does not explicitly state when to use this tool versus alternatives or when not to use it, nor does it mention any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rale_commandA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds that every call surfaces a toast in Dev Studio, complementing the annotations (destructiveHint=true). It reinforces the destructive nature with examples.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, each adding critical information: the tool's purpose, a key alternative, and a side effect. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While it covers the main use case and destructive intent, it lacks details on return values or error handling. For a tool with no output schema and destructive potential, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 67%, the description adds little beyond restating command purpose. It does not elaborate on 'args' or 'device' parameters beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it runs any built-in RALE command, including destructive ones, and points to list_rale_builtins for the catalog, distinguishing it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly advises using list_rale_builtins for the catalog and hints at destructive use, but does not fully specify when to use this tool over other sibling tools like get_action_schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_connectionA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the description adds value by specifying the return content (reachability + basic device info) and the open-world condition. No contradictions found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long with zero fluff. It is front-loaded with the core purpose and immediately provides the key condition. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple probe tool with rich annotations and a well-documented schema, the description covers purpose, return info, and a usage condition. It lacks mention of error cases or response structure, but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds extra semantics beyond the schema's parameter description: 'Omit to use the focused device.' This clarifies default behavior, which is useful for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Probe a device IP for ECP availability') and outcome ('Returns reachability + basic device info'). However, it does not explicitly distinguish itself from sibling tools like 'probe_bridge' or 'network_inspector_status', which may have overlapping purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a key usage condition ('Does not require the device to have a tab open'), but lacks guidance on when to avoid this tool or alternatives. The condition is helpful for context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ecp_postA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false, covering the side-effecting nature. The description repeats 'Side-effecting' but adds no additional behavioral context such as consequences of misuse or rate limits. Minimal added value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficiently crafted sentences: first states the core action with an example, second provides a cautionary note and alternative. No filler, front-loaded with the essential purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no output schema, and informative annotations, the description covers the core function, safety warning, and alternative usage. Some detail about endpoint validation or response behavior is missing, but overall sufficient for a straightforward POST tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% with only the 'device' parameter described. The description adds an example endpoint value ('/sgrendezvous/track') but does not clarify the format or constraints of the 'endpoint' parameter. Some help, but incomplete for the undocumented parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('POST to an arbitrary ECP endpoint') with a concrete example ('e.g., /sgrendezvous/track'), distinguishing it from other tools. It also mentions a safer alternative (list_post_presets), reinforcing its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises when to use the tool ('agents should use list_post_presets for safe defaults'), providing clear guidance on appropriate use cases and a direct alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• network_inspector_analyzeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, and idempotentHint. The description adds behavioral context that the tool aggregates the buffer into rollups, requires Network Inspector enabled, and returns summary counts. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph of three sentences with no fluff. It front-loads the main action and includes essential details like filter compatibility and prerequisites.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately explains the return values (counts by various categories) and notes the prerequisite (Network Inspector enabled). Although there is no output schema, this covers the main aspects needed for an agent to decide to use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description merely repeats that the tool accepts the same optional filters as a sibling tool. It does not add deeper semantic meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'aggregate' and the resource 'captured buffer into hotspots and rollups', listing specific outputs like counts by event type, HTTP status class, etc. It distinguishes itself from sibling tools like network_inspector_list_events by stating its purpose to 'orient on a session before drilling into individual events'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to orient on a session before drilling into individual events', providing clear context for when to use it. It also notes that it accepts the same filters as network_inspector_list_events, hinting at sibling differentiation. However, it does not explicitly state when not to use it or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_script_to_builderA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds context beyond annotations: does not auto-run, runs same validation as validate_script. No contradiction with annotations. Annotations already provide idempotent and open world hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with core purpose, then usage guidelines. Two sentences plus a longer instruction, minimal waste. Could be slightly tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 params, no output schema, and annotations, the description covers purpose, usage, and parameter shape. No output schema needed; completeness adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. Description mentions script and device parameters briefly but adds no new semantic information beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states tool sends a validated Action Script to Dev Studio Builder for human review, distinguishes from siblings by mentioning direct ops as alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (multi-step/conditional/saved-or-reviewed flows) and when not (single deterministic action), listing alternative direct ops.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capability_bundleA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior. The description adds value by recommending caching and noting an alternative resource URL, which are not covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences. The first defines content, the second provides usage guidance. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description fully covers what the tool returns (listing all types), when to use it (before scripting), and additional context (caching, alternative resource).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so description does not need to add param details. The baseline score of 4 applies as there is nothing to improve.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a single payload of all static capabilities, listing specific types (actions, vocabularies, etc.). This distinguishes it from sibling tools like get_action_schema which return individual items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises loading once before authoring scripts and caching, providing clear usage context. It doesn't explicitly exclude use cases, but the guidance is sufficient for a data-fetching tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• network_inspector_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare safe read-only behavior. Description adds useful context on metrics reported and the condition that reads are useless until ready is true, which is valuable beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence packs all essential info (purpose, fields, usage order). No superfluous text, but could be broken into two sentences for readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description enumerates returned fields and remediation logic, providing sufficient completeness for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. Description explains the return fields (enabled, capturing, etc.) compensating for lack of output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool reports Network Inspector status including specific fields like enabled, capturing, client count, etc. It distinguishes from sibling network_inspector_* tools by being the status check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call this first before other network_inspector_* tools and provides conditional guidance on using notice/remediation when ready is false.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_bridgeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint. Description adds context about return format and session usage, but behavioral traits are mostly covered by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with clear structure; front-loads return format and essential usage instructions with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description fully explains return format and session flow. Complete for a probe tool given low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100%. Description adds no param info, but baseline is 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool returns bridge status and distinguishes it from sibling tools by noting that once live=true, other tools can be used freely.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call once per session before first bridge-dependent tool and contrasts when to use direct ops vs re-probing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sideloadA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (destructiveHint true), the description details that the tool replaces any currently sideloaded Dev App, explains temp file cleanup for contentBase64, and warns about sandbox path errors. This adds significant context beyond structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear main statement followed by method details. However, it is slightly verbose; some sentences could be tightened without losing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the process well, including cleanup and error behavior for wrong paths. However, it does not explain the return value or success/failure indicators, which would help the agent understand the response. Given the complexity and lack of output schema, it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds meaningful inter-parameter relationships (mutual exclusivity of filePath and contentBase64) and operational context (temp file handling, sandbox path detection). This goes above the baseline for fully documented schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool uploads and installs a .zip channel package, specifying the verb 'Upload and install' and the resource 'channel package'. It distinguishes from siblings by noting the destructive nature and providing two methods for different environments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance for each parameter combination: filePath for local execution, contentBase64 for remote agent sandboxes. It also explains when password is optional, making usage conditions very clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• telnet_disconnectA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds idempotency detail (returns { connected: false, already: true } when no session), which goes beyond annotations (idempotentHint). No contradictions with annotations (readOnlyHint, destructiveHint).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler. First sentence states purpose, second describes behavior, third gives usage. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool (one optional parameter, no output schema), the description covers purpose, behavior, and usage completely. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter. Description adds value by explaining that omitting device uses the focused tab, which is helpful for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it closes the BrightScript debug console (TCP 8085) mimicking the Disconnect button. It is specific about the resource and action, distinguishing it from sibling tools like telnet_connect or other network tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use it to release the 8085 socket for another tool to attach or to stop log accumulation. No explicit when-not-to-use, but context is clear enough for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• network_inspector_get_event_detailA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint, openWorldHint, idempotentHint. The description adds behavioral details beyond annotations: body truncation cap (maxBodyChars default 4096), warnings on truncation, includeFullBody override, and no body events potentially returning 404. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. Front-loaded with action and key details. Very efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters and no output schema, the description covers all critical aspects: what it retrieves, truncation behavior, edge cases, prerequisites. It is complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description adds significant context: id is from list_events, maxBodyChars default 4096, includeFullBody overrides truncation behavior. Provides value beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Fetch the full headers and body for one captured event by id' and specifies the resource (network event). It distinguishes itself from siblings by referencing 'network_inspector_list_events' as the source of the id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the prerequisite 'Requires Network Inspector enabled' and notes edge cases (DNS/TLS/TCP events may return 404). However, it does not explicitly state when to use this tool versus alternatives (e.g., list_events for listing) but implies the use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.