Server Quality Checklist

Tool Scores

• abort_taskB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it aborts the running task but doesn't disclose consequences (are results lost? Is it reversible? Does it need admin privileges?). An abort operation likely has destructive implications that are not surfaced, and there's no annotation to cover the safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with zero wasted words. It's efficient and front-loaded. Slight deduction only because it's arguably under-specified rather than optimally concise for a potentially destructive operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description is reasonably complete on the surface. However, the absence of any behavioral context about side effects or prerequisites of an abort operation is a notable gap for a destructive-sounding tool. It's adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters and no schema to document, so the schema coverage is trivially 100%. With no parameters, there is nothing for the description to explain, making the baseline 4 for parameter semantics appropriate since no compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Abort') with a clear resource ('the currently running Agent task'). It clearly states what the tool does. It lacks explicit sibling differentiation, though the sibling list shows related task tools (run_agent_task, get_task_result) where this abort action is contextually distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states what it does but gives no guidance on when to use it versus alternatives or what conditions apply. It doesn't specify whether it applies only to tasks started by this system, what happens to partial results, or whether running is a prerequisite. No exclusions or explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_memoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral transparency. It states the write/persist nature ('into persistent memory'), but does not disclose whether the operation is additive-only, how it interacts with existing memories of the same title, whether duplicates are overwritten or stacked, storage limits, or any side effects. Beyond the schema's own field hints, little additional behavior is revealed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that conveys the core action and purpose without waste. Any behavioral or usage detail is absent, but as written it is maximally efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple 3-param write tool with 100% schema coverage and no nested objects, so it doesn't demand much. However, with no annotations and no usage exclusions, it would benefit from clarifying memory-management behavior (e.g., whether same-title writes override or append, and typical use cases vs search_memory). Adequate but with gaps given the sibling tooling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all three parameters (title, content, keywords) have inline schema documentation. The description adds the 'under 200 chars recommended' guidance via the schema, not the tool description itself. The description text adds little beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'Save' plus the resource 'semantic memory' clearly states the tool's action. The parenthetical 'reusable knowledge' usefully clarifies what a semantic memory means. It's distinguishable from siblings like search_memory and delete_memory, though it doesn't explicitly call out any sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes this is for 'reusable knowledge' saved into persistent memory, implying it's for knowledge that should persist. However, there is no explicit guidance on when NOT to use it, when to prefer alternatives, or what kind of content is appropriate beyond the persistence framing. It lacks excluded cases (e.g., ephemeral data).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoryB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint=true annotation already declares this as a safe read operation, so the bar for transparency is lower. The description confirms it's a search (non-mutating) operation, consistent with the annotation. However, it doesn't disclose what happens when no matches are found, whether results are ordered, or anything about the result format, which would be useful given there's no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that packs in the tool's purpose and the semantic/episodic distinction. It's front-loaded with the core action ('search the persistent memory') followed by clarifying detail. No wasted words, though it could arguably say more about result behavior without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with readOnlyHint, 100% schema coverage, and no output schema, the description provides adequate but not rich context. It defines the two memory types, which is helpful. However, it doesn't describe what the results look like (the return format is undocumented), and it doesn't mention pagination or ordering behavior. A search tool with a limit parameter would benefit from describing what happens with results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters. The description adds the semantic vs. episodic distinction and mentions keyword search, which aligns with the query parameter's 'keyword query' description. However, the description doesn't add meaningful semantic detail beyond what the schema provides—it clarifies what the two memory types mean ('learned experience' vs 'task history') but doesn't enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (search), resource (persistent memory), and scope (keyword-based search). It also distinguishes between semantic memories and episodic records, which adds specificity. It doesn't explicitly differentiate from sibling tools like delete_memory or save_memory, but the verbs alone make the distinction obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is for searching memory contents but doesn't provide explicit when-to-use guidance or contrast with alternatives. The mention of both semantic and episodic types gives some context, but there's no statement about when to search one type versus the other, nor guidance about when to use run_agent_task or other task-related tools instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_memoryA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the operation targets either semantic or episodic memory types, which is useful scope information. However, it doesn't disclose whether deletion is permanent/irreversible, whether it requires any permissions, or whether related/referenced memories are also affected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with zero wasted words. It front-loads the primary verb and resource, then scopes with the semantic/episodic distinction and id requirement. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter destructive operation with no output schema, the description is reasonably complete. However, as a delete operation with zero annotations, it would benefit from stating irreversibility or idempotency behavior. The mention of 'semantic or episodic' types is genuinely helpful context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'id' parameter. The description adds minimal value by confirming the id refers to what search_memory/save_memory return (already stated in the schema). With full coverage, the baseline of 3 is appropriate with no significant extra meaning added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Delete one memory entry (semantic or episodic) by its id' - a specific verb (delete) and resource (memory entry), with clarity that it handles both semantic and episodic types. It differentiates from siblings like save_memory and search_memory by describing the write/delete operation, though it doesn't explicitly contrast with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'by its id' and mentioning both semantic and episodic memory types, indicating when this applies. However, it doesn't provide explicit when-to-use vs alternative guidance, nor does it address edge cases like what happens when the operations is attempted on a missing/invalid id.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_latest_taskA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, so the safety profile is already covered. The description adds scope context ('most recent', 'current conversation') beyond annotations. However, it doesn't disclose what happens if no task exists in the conversation (error, null result, etc.), and being read-only is largely implied by the annotation already.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly-scoped sentence with zero waste. Every word earns its place — verb, resource, and the two scope qualifiers ('most recent', 'current conversation') all carry meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 0-parameter, read-only, no-output-schema tool, the description is reasonably complete. The main gap is behavioral: what happens on an empty conversation or when no agent task has run, and whether 'task result' refers to run_agent_task or get_task_result outputs. Some ambiguity remains around what constitutes a 'task result' and its relationship to sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and schema coverage at 100%, the schema fully documents everything. No parameter information is needed in the description since there are no parameters to explain. Baseline 4 for a 0-param tool is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Get the most recent Agent task result from the current conversation' — a specific verb+resource ('get... task result') with clarifying scope ('most recent', 'current conversation'). It's clear about what it does, though it doesn't explicitly distinguish from sibling get_task_result, relying on the 'most recent' qualifier to differentiate scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'from the current conversation' provides some context for when it applies, and having sibling get_task_result implies there's a scoping distinction (latest vs specific). However, there's no explicit guidance on when to choose this over get_task_result, when the conversation context matters, or any prerequisites (e.g., must a task have been run first?).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_agent_taskA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the agent autonomously plans and executes multiple steps using screen vision and touch controls, which is important behavioral information (the agent has agency and may perform actions beyond a simple explicit one). The annotations already declare destructiveHint=true and readOnlyHint=false, which the description aligns with. It adds context that the agent is autonomous and may take multiple actions, which is valuable beyond the raw annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence plus an example in the schema. It's front-loaded with the core purpose and adds the key behavioral detail about autonomous planning. Efficient and earns its space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 well-documented parameters, no output schema, and good annotations, the description covers the essential purpose, the autonomous execution mode, and the execution mechanism. It could mention what the return value looks like (task ID, result, etc.) since there's no output schema, but given the 100% schema coverage and sibling context, it's reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already fully documents both parameters (goal and timeoutMs) with descriptions. The description adds some context by showing a concrete example ('Open WeChat and send hello to John') in the schema, and mentions timeouts indirectly. Baseline of 3 is appropriate since the schema carries the parameter documentation burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs an AI agent task on the connected phone, explaining it plans and executes steps using screen vision and touch controls. It uses a specific verb+resource combination ('run agent task'), which distinguishes it from siblings like take_screenshot or get_device_status. However, it doesn't explicitly differentiate itself from related task tools like abort_task or get_latest_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys it's for multi-step goals achieved via vision and touch, implying usage for complex tasks. However, there's no explicit guidance on when to use this vs alternatives like take_screenshot or when not to use it. The distinction from simpler direct-action tools is implied but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_device_statusA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description aligns with this by framing the operation as 'Get information' — no contradiction. The description adds value by specifying what kind of information is returned (model, brand, OS version, etc.), which helps the agent anticipate output. It doesn't cover edge cases like device-not-connected errors, but for a read-only info tool, the coverage is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point. It's front-loaded with the core purpose and enumerates example fields for concreteness without padding. Slightly more structure (e.g., a separate note on return behavior) could help, but it's appropriately lean for a zero-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, a readOnlyHint annotation, no output schema, and low conceptual complexity, the description is largely sufficient. What's missing: no explicit mention of failure modes (e.g., what happens when no device is connected), but for a zero-param info retrieval tool, the description covers the essentials well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and with no params, the baseline is 4 per the rubric. There are no parameters to document, so semantic gaps don't apply. The description appropriately explains what the tool retrieves rather than trying to document nonexistent parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Get information about the connected phone device') and enumerates example fields (model, brand, OS version), making the purpose clear. However, it doesn't explicitly distinguish itself from sibling tools, though the sibling set (memory, tasks, screenshots) is clearly different enough that no confusion is likely.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (checking device details) but provides no explicit when-to-use guidance or exclusions. With 8 siblings spanning memory, task, and screen operations, the absence of any 'use this for X, not Y' type guidance leaves some room for ambiguity, though the tool's domain (device info) is fairly self-apparent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_task_resultA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, which the description aligns with (a 'get' operation is consistent). The description adds context that the task must be 'completed' for a result to exist, implying that incomplete/running tasks won't yield results via this tool. This is a meaningful behavioral constraint beyond the annotation. Siblings like abort_task and run_agent_task suggest a task lifecycle where this tool is the terminal read step.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, compact sentence that states exactly what the tool does with no filler. Every word earns its place. This is appropriately front-loaded and minimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only result-fetch tool with a fully documented schema (100% coverage, no output schema), the description plus annotations cover the essential contract: what it returns (result of a completed task), key input (taskId), and optional behavior (includeSteps with default). The absence of an output schema reduces the descriptive burden. A minor gap is not noting what happens for a task in progress or an invalid ID, but for a simple fetch tool this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters documented: taskId is described as 'returned by run_agent_task' and includeSteps has a default of true. The description itself adds no parameter-specific detail beyond 'by its task ID,' relying on the schema. Baseline 3 is appropriate since the schema does the heavy lifting and explicitly documents both parameters well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Get the detailed result of a completed Agent task by its task ID.' It states the resource (task result), the action (get), and the identifier (task ID). It doesn't differentiate from siblings like get_latest_task, which retrieves tasks but for a different scope (latest vs. by ID). A small mention of 'completed' adds scope but no explicit sibling distinction, so 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving completed task results by task ID, but doesn't explicitly state when to use this vs. get_latest_task or when not to use it. No alternative tools are named. The context (completed task, task ID from run_agent_task) is implied by the parameter description in the schema rather than stated in the tool description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• take_screenshotA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already declares this is a safe read operation, so the bar is lower. The description adds that it returns a PNG image, which is a useful behavioral detail. However, it does not disclose potential limitations such as screen lock state, permission requirements, or what happens if no phone is connected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with zero wasted words. It front-loads the core action ('Take a screenshot') and follows with the output format. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless, read-only tool with a clear purpose and stated output format (PNG), the description is reasonably complete. There is no output schema, but the description addresses the return type. Minor gaps around failure conditions exist, but they don't substantially impede correct tool usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters, so the schema carries no burden and the description doesn't need to explain any. With no parameters to document, the baseline of 4 applies. The description correctly focuses on behavior and output rather than parameter details that don't exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (take a screenshot), the target resource (connected phone screen), and the output (PNG image). It distinguishes itself from siblings—none of which relate to screen capture—so there is no ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when a screenshot of the phone is needed) but does not explicitly name alternatives or exclusion conditions. There is no guidance about prerequisites like device connection state, which the sibling get_device_status might help verify. However, the purpose is clear enough that an agent can infer appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.