Server Quality Checklist

Tool Scores

• screenshotC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It does disclose that the tool falls back between two mechanisms, which is a useful internals detail. However, it omits critical behavioral aspects: whether the screenshot is returned as a file, base64, or URL; whether it blocks or is asynchronous; failure modes; and any side effects (e.g., requiring user consent). This is insufficient for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences. It front-loads the primary purpose and then adds a relevant implementation note. No fluff or redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description should at least hint at the return format (image? path?), common error scenarios, and any side effects. It also doesn't clarify whether this is a still capture vs. a stream (which the sibling 'screen_stream' covers). Given the simplicity, the gaps are significant and would impair an agent's ability to invoke and interpret results correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with 'device' described as 'Device ID or name'. The description adds no additional meaning or constraints (e.g., format, lookup behavior). Since the schema fully covers the parameter, a baseline 3 applies; the description doesn't add value beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Capture') and resource ('a screenshot from a device'), specifying the tool's core function. However, it does not explicitly differentiate from the sibling tool 'screen_stream', which could be ambiguous for an agent deciding between capturing a still vs. streaming. The purpose is still unambiguous enough for a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like screen_stream or call. It does not mention prerequisites (e.g., device must be on, permissions needed) or any conditions that would make this tool preferable. The implementation detail about MediaProjection/AccessibilityService is not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• callB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It mentions the pairing prerequisite and internal routing through MobileGatewayService, but it does not warn that invocations can be mutating or disruptive (e.g., tap, swipe, launch, clipboard) and it does not describe result shapes or error behavior. This is a meaningful gap for a tool that can perform device-level actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then gives a useful tool list plus a docs reference. The 21-item tool list is long but earns its place because it helps the agent understand the dispatch surface. It is structured and free of filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the generic nature of the dispatcher and the absence of an output schema, the description compensates well with a comprehensive tool list, per-tool examples, a docs pointer, and the pairing precondition. It does not explicitly state that the return value is whatever the underlying device tool returns, which is a minor omission, but overall the agent has enough context to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description goes further by giving tool-specific examples and required arguments for many device tools, such as computer.tap needing x,y and computer.key accepting back|home|enter|recents. This adds real selection value beyond the raw schema; the opaque 'arguments' object is the main residual gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource relationship: 'Execute a tool on a paired Android device.' It also enumerates the device tools that can be invoked through this dispatcher, giving the agent a concrete sense of what the tool covers. It does not deeply differentiate from siblings, but the generic dispatch role is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by listing available device tools and pointing to a full reference doc, but it never explicitly says when to prefer this tool over sibling tools like screenshot or batch. There is no 'when not to use' guidance or comparison to alternatives, leaving some routing judgment to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full behavioral burden. It discloses two useful behaviors: execution is concurrent and results are returned per device. However, it doesn't cover failure handling, partial success, ordering, or whether side effects depend on the selected tool, leaving meaningful gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences convey the core operation and return behavior with no filler. The action is front-loaded and every word adds meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generic dispatcher without output schema or annotations, the description is adequate but incomplete. It covers concurrency and per-device results, but doesn't specify valid values for 'tool', how 'arguments' map to the selected tool, or behavior on partial failures, so an agent is left to infer important invocation details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema describes all three parameters with 100% coverage, so the schema does the heavy lifting. The description adds only context about multiple devices and per-device results, not new parameter-level semantics, which aligns with the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Run an operation') and resource ('against multiple devices concurrently'), and mentions per-device results. This distinguishes it from single-device siblings like call or get_device, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes a clear selection context: use this tool when an operation must be applied to multiple devices concurrently. It doesn't state exclusions or name alternative tools, but the multi-device framing is an explicit usage signal, so it's more than implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deviceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the kind of data returned (capabilities, connection state, foreground app), which is useful, but it does not state that the operation is read-only, mention error conditions, or note whether the device must be paired/online.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the action, and every clause adds value by listing the specific details returned. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter getter with no output schema, the description is nearly sufficient. It names the key return fields, but omits possible error handling or device availability constraints. Overall, an agent can call this correctly with minimal guesswork.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already describes the device parameter with examples. The description adds no additional semantic information about the parameter, matching the baseline for complete schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Get') and resource ('a specific device'), and distinguishes itself from list_devices by targeting one device rather than all. The mention of capabilities, connection state, and foreground app makes the function unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need details on a single known device, contrasting with listing, but it does not explicitly name alternatives or state when not to use it. No exclusions or prerequisites are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unpairA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the primary effect (removing pairing) and the consequence (device becomes uncontrollable until re-paired), which is useful. However, it does not mention potential side effects, error conditions, or whether the operation is idempotent, leaving some behavior unexplored.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two short sentences—with the primary action front-loaded and the consequence following immediately. Every word earns its place, and there is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one required parameter and no output schema, the description covers the essential behavior and outcome. It explains the action and the need to re-pair, which is sufficient for an agent to understand the tool's purpose and likely effects. Minor gaps like error handling or prerequisites are not critical here.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the parameter with a clear description ('Device ID or name to unpair'). The tool description adds no further parameter-specific details beyond what the schema already provides, so this is a baseline 3 as per high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Remove pairing for a device.' It uses a specific verb and resource, and the added consequence ('The device will need to be paired again before it can be controlled.') distinguishes it from sibling tools like pair and reinforces the reverse operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when you want to remove a device's pairing), but it does not explicitly mention alternatives or conditions for selection. It lacks guidance on when one might prefer unpair over other sibling tools, though the context makes it fairly obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_devicesA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It makes the read-only nature clear ('List'), explains what will be returned ('discovered and paired devices'), and discloses prerequisites and failure conditions such as installation, accessibility service, and same-network requirements. This is strong, though it could add explicit statements about no side effects or return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main action and then adds a clearly enumerated troubleshooting list. It is somewhat longer than strictly necessary, but every sentence serves a purpose and the numbered steps make it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter, this description is quite complete: it covers scope, setup prerequisites, and failure behavior. It does not describe the output format, but the absence of an output schema and the straightforward 'lists devices' meaning reduce the severity of that gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the single optional status parameter is already well documented in the schema. The description adds no parameter-specific detail, but it does not need to because the schema handles it, yielding the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'List all discovered and paired Android devices running the FreeCode Agent app.' It clearly states the tool's scope, but it does not explicitly differentiate this from the sibling get_device tool, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: to list discovered/paired devices. It even provides troubleshooting steps when no devices appear, which is useful practical guidance. However, it never names alternatives or says when not to use this tool, so it is not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pairA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it does meaningfully: it states the prerequisite state, discloses both code modes, and reveals that pairing requires an interactive user confirmation on the phone. It falls short of a 5 by not addressing blocking/timeout behavior or what the tool returns (particularly the auto-generated code).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with zero waste: action, prerequisite, then the two-path flow logic folded elegantly into one sentence. The main verb+object is front-loaded, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter tool with no annotations and no output schema, the description covers the prerequisites and both operational paths well. The main gap is that it never states how the agent obtains the auto-generated code to relay to the user (presumably via the tool response), which is an important post-invocation detail given there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3 — the schema already documents both parameters. The description reinforces the code parameter's decision rule ('if a code was generated on the device, pass it; otherwise one is generated'), which adds marginal value, but it contributes nothing new for the device parameter beyond what the schema states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Initiate pairing with a discovered Android device.' This clearly distinguishes it from its siblings — unpair is the reverse operation, list_devices/get_device are read-only, and call/batch/screenshot/screen_stream are operations that presuppose pairing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when the tool applies: the device must already be discovered and running the FreeCode Agent app, which implies the list_devices step precedes this. It also explains the two invocation flows (pass an existing code vs. auto-generate). However, it never explicitly names alternatives or exclusions, such as pointing to unpair as the reverse operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screen_streamA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It discloses that polling returns a sliding window of the last few frames (oldest→newest), implies persistence, and mentions the stop action to end the stream. It does not detail side effects or cleanup beyond stopping, but the key behavioral aspects are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense sentence that front-loads the purpose and then explains each action. It uses parentheses effectively to clarify the poll behavior. It is concise without wasting words, though the lengthy sentence could be slightly more scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the three actions and the polling semantics in detail. It does not describe the exact return format (e.g., binary image data or URLs) or error conditions, but given the tool's complexity and the absence of an output schema, it provides enough for an agent to use it correctly for typical streaming scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds significant meaning to the 'action' parameter by explaining the semantics of each value and the sliding-window behavior. For 'device' and 'format', it adds little beyond the schema, but the core parameter is well elaborated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (start, poll, stop) and the resource (persistent screen stream). It distinguishes from siblings like screenshot, which is a one-time capture, by emphasizing the persistent nature and the sliding-window polling behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to use each action (start, poll, stop) and provides context for poll being called repeatedly to watch self-changing screens. It does not explicitly name alternatives (e.g., screenshot) or state when not to use the tool, but the action-specific guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.