Server Quality Checklist

Tool Scores

• get_device_infoA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the output format (XML) but does not mention that the operation is read-only, side-effect-free, or idempotent. Lacks depth for a tool with zero annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It front-loads the purpose and format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so the description need not explain return values. The description covers the core purpose and format. However, lacking usage guidelines or behavioral notes makes it slightly incomplete for a completely annotation-free tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema description coverage is 100%. Per the rules, baseline is 4. The description adds no parameter information, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves basic device information (model, software version) as XML. It distinguishes from sibling tools like 'discover_tv' or 'launch_app' which have different purposes, but 'etc.' slightly reduces specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. While siblings are functionally distinct, the description does not provide any context on prerequisites or when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_onA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states the intended behavior (powers on the TV) but does not disclose any additional behavioral traits such as potential delays, error scenarios, or side effects. Since no annotations are provided, the description carries the full burden, and while the core action is clear, more detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly communicates the tool's purpose with no unnecessary words. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (zero parameters, straightforward action) and the presence of an output schema, the description is minimally adequate. However, it omits useful context such as prerequisites (TV off), expected response, or potential side effects. For a trivial action, this may be acceptable, but more context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the input schema is fully covered (100%). The description does not need to add parameter information. With zero parameters, a baseline score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (power on) and the resource (the TV). It is distinct from sibling tools like discover_tv, get_device_info, launch_app, list_apps, and press_key, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool vs. alternatives. For example, it does not mention that the TV should be off before calling, or suggest checking the TV state first with get_device_info. No exclusion criteria or context are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_appsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only mentions what the tool returns (a formatted list). It does not disclose any additional behavioral traits such as caching, dependencies, or performance characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description adequately explains what is returned and its utility. It is missing potential limitations but is sufficient for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the baseline is 4. The description adds no parameter information, but this is acceptable since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'lists' and the resource 'all available apps and their corresponding Roku channel IDs', and distinguishes it from sibling tool 'launch_app' by explaining how the output can be used with that function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining app IDs to use with launch_app(), but does not explicitly state when to use this tool versus alternatives like discover_tv or get_device_info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• press_keyA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavior. It only states it simulates a press, lacking details on prerequisites (e.g., active TV connection), side effects, or error conditions. Minimal behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise paragraph with clear structure (description, Args list). Every sentence adds value. Minor improvement could be more structured formatting for AI parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers the core action adequately. Missing return value description despite output schema existing, and no mention of prerequisites like device connection. Adequate but with gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds extensive meaning: lists examples, common keys, and special keys. This compensates well, though could include case sensitivity or exhaustive list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it simulates a single button press on the TV remote, with specific verb and resource. It distinguishes from sibling tools like launch_app and power_on by focusing on key presses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides examples of common and special keys, giving clear context for use. However, it does not explicitly exclude alternatives like power_on for power keys, but the purpose is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch_appA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description bears full responsibility for behavioral disclosure. It notes case-insensitivity but does not mention error handling (e.g., if app not found), return values, or side effects (e.g., if TV powers on automatically). This leaves gaps for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three brief sentences), uses a clear 'Args:' structure, and contains no extraneous information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists (not shown but present), the description need not detail return values. It sufficiently explains the app_name parameter and references list_apps. Lacks mention of prerequisites (e.g., TV power state) but is otherwise complete for a simple launch action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter app_name is enriched with examples (Netflix, YouTube), case-insensitivity note, and a reference to list_apps. Since schema description coverage is 0%, the description provides all necessary semantic context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it launches an application on the TV by name, which is specific and distinguishes from sibling tools like list_apps (listing apps) and press_key (key input).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises using list_apps() to see available apps, providing useful prerequisite guidance. However, it does not explicitly mention when to use this tool versus alternatives or conditions like whether the TV must be on.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_tvA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it updates the active TV to the first found device, which is a notable side effect. With no annotations, the description carries the full burden and does so well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded action and side effect, then usage scenario. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, side effect, and usage. Output schema exists to detail return values. Could mention potential delays from network scanning, but not necessary for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the description adds no param info beyond the empty schema. Baseline for 0 params is 4, and the schema coverage is trivially 100%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it scans for Roku TVs via SSDP and returns discovered devices, with a distinct side effect of updating the active TV. This differentiates it from sibling tools like get_device_info or launch_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies it is useful when HOST_IP is not configured or IP may have changed. No direct exclusion of when not to use, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.