Server Quality Checklist

Tool Scores

• annotate_closeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral transparency burden. It does disclose an important side-effect: 'Marks are kept on disk,' which assures persistence. However, it does not mention idempotency, whether all browser instances are closed, or any other behavioral traits. The single guarantee is useful but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two short sentences, each providing distinct value. 'Close the annotation browser' states the action, and 'Marks are kept on disk' clarifies a key behavioral guarantee. There is zero irrelevant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter tool with no annotations and no output schema, the description covers the essential purpose and a critical side-effect. It could be improved with a note about idempotency or what happens if already closed, but the core context is sufficiently provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already covers this (100% coverage trivially). The description adds no parameter information, but none is needed. Per rubric, zero parameters yields a baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action with a specific resource: 'Close the annotation browser.' This distinguishes it from sibling tools like annotate_open and annotate_clear. The purpose is immediately obvious and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives. It implies closure after annotation work, but does not provide context, exclusions, or mention of sibling tools. The description does not help an agent decide between this and, say, annotate_clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_screenshotA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosure. It effectively communicates the capture behavior (toolbar hidden, drawings visible, optional crop) but does not describe the output format (e.g., image data or path) or any side effects, which are relevant for a tool without an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact two sentences, front-loaded with the primary action and followed by a clear optional modifier. Every clause serves a purpose, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two optional parameters and no output schema, the description covers the main behavior and the crop option. However, it does not clarify the interaction between markId and fullPage (e.g., precedence if both are provided) or what the screenshot output actually is, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds modest context for markId (its purpose for inspecting a squiggle) but adds nothing for fullPage beyond what the schema states. It does not need to compensate since the schema is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb + resource ('Screenshot the current page') and adds unique capabilities ('drawings visible but the toolbar hidden', 'crop tightly around that mark'). This clearly distinguishes it from sibling tools like annotate_list or annotate_resolve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The statement 'the fastest way to see what a squiggle actually meant' provides clear context for when to use the tool, and the markId parameter implies a use case. However, it does not explicitly mention alternatives or say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_listA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It reveals that output is a 'readable briefing, grouped by page' and implies non-blocking behavior ('without waiting for a send'). However, it does not explicitly state that it is read-only or describe any side effects, permissions, or pagination behavior. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences, each serving a distinct purpose: the first states what the tool does, and the second advises when to use it. No wasted words, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (2 optional params, no output schema), the description covers the essential purpose, usage, and some output characteristics. It could be more specific about the return format or filtering semantics, but for a list tool with good schema coverage, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% coverage with meaningful descriptions for both 'status' and 'pathFilter'. The description adds no additional meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List annotation marks as a readable briefing, grouped by page,' which is a specific verb+resource+output format. It distinguishes itself from sibling tools like annotate_inbox or annotate_resolve by focusing on the listing operation. Purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit use case: 'Use this to check what is pending without waiting for a send.' This gives clear guidance on when to invoke it, distinguishing it from tools that send or resolve annotations. However, it does not explicitly mention when not to use it or name alternative tools, so it stops short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_inboxA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a non-blocking read operation ('without waiting', 're-read'), which implies no mutation. It doesn't describe return values or edge cases, but for a zero-parameter read tool, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler: the first states the action and a key behavioral trait, the second gives the when-to-use context. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with no parameters and no output schema, the description covers purpose, usage, and timing. It doesn't define 'Send batch' or return format, but the explanatory clause and non-waiting hint make it reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters and the schema coverage is 100%, so the description doesn't need to explain inputs. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb ('re-read') and resource ('most recent Send batch'), and adds an explanatory clause about its use case. It distinguishes itself from sibling tools by focusing on the inbox rather than screenshots or resolution, though the term 'Send batch' is left undefined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a concrete scenario ('if the conversation moved on and you need the original request again') and highlights the non-waiting behavior, implying this is faster than waiting. It doesn't explicitly name alternatives, but the context is clear enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_clearA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden. It discloses permanence ('permanently') and the dangerous bulk behavior ('with no ids, deletes everything'), which is critical for a delete operation. It does not mention auth or undo, but the key risks are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff. The first sentence states the action, and the second clarifies the critical edge case. Excellent front-loading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional param, no output schema), the description fully covers its purpose, behavior, and the critical default case. No missing information that would impede correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single 'ids' parameter is already fully described in the schema ('Specific mark ids. Omit to clear all.'), so schema coverage is 100%. The tool description adds no extra parameter semantics beyond what the schema provides, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Delete') and resource ('marks'), and clearly distinguishes behavior when ids are provided versus omitted. This differentiates it from sibling tools like annotate_list or annotate_resolve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context on how to use the tool: provide ids for specific marks, omit ids to clear all. It does not explicitly name alternatives or exclusions, but the parameter guidance is sufficient for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_resolveA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states a key side effect: 'They stay visible on the page but dim' and explains the rationale ('so the human can see what you handled'). This goes beyond a simple state change and helps the agent anticipate the visual outcome. It does not cover reversibility or error behavior, but that is acceptable for a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and the second sentence adds a valuable behavioral note about visibility and dimming. There is no wasted verbiage; every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is complete. It explains what the tool does, when to use it, and the visible after-effect. The input schema fully documents the parameter, and sibling tool context implies a family of annotate operations. No critical information is missing for an agent to invoke this correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the only parameter 'ids' with description 'Mark ids to resolve.' The tool description adds some conceptual context by referring to 'items' and 'acted on them,' but it doesn't provide additional detail beyond what the schema already states. Since schema coverage is 100%, a baseline of 3 is appropriate; the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+object structure: 'Mark items as done' clearly states the action and target. It distinguishes itself from sibling tools like annotate_clear (which likely removes annotations) by focusing on a 'done' state, and the phrase 'once you have acted on them' ties it to a workflow step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear contextual guidance: use after acting on an item ('once you have acted on them'). It does not explicitly mention alternatives or when not to use it, but the context is sufficiently clear for an agent to decide when to invoke it. It implies a sequential workflow without naming sibling exclusion cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_openA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: headed Chrome, persistent profile, automatic toolbar injection, optional URL navigation, and the requirement to follow with annotate_wait. It also explains the 'parked' mechanism, adding valuable context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tight sentences, front-loaded with the core action. The additional sentences explain the injected toolbar and the mandatory follow-up with annotate_wait, each earning its place. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's nature (opening a browser for human interaction), the description covers the essential aspects: what it does, sequence constraints, and why the wait is necessary. It doesn't need to explain return values since there's no output schema. The description provides enough context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'url', so the schema already documents its meaning. The description only says 'optionally navigate to a URL', which adds no new information beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool opens the annotation browser (headed Chrome, persistent profile) and optionally navigates to a URL. The verb 'Open' is specific, and the resource and behavior distinguish it from sibling tools like annotate_inbox or annotate_screenshot.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given: 'Call this first' and 'ALWAYS follow this with annotate_wait in the same turn'. It explains why this ordering is necessary (human expects Send to reach you only while something is parked), providing clear when-to-use vs. alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• annotate_waitA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: parking is free and token-free, clients may background the call after two minutes, long parks are kept alive with pings, there is a per-server timeout ceiling, and the overlay shows a live indicator. This goes far beyond minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact paragraph where every sentence earns its place: purpose, rationale, usage timing, client behavior, and UI feedback. It is front-loaded with the core purpose and contains no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's interactive waiting nature, the description covers what happens when no one is parked, what the caller experiences (background task), how long waits are kept alive, and what the human sees. It provides the necessary operational context despite no output schema or annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the single timeoutSeconds parameter with rich detail on defaults, max, and behavior. The description itself adds no new parametric meaning beyond restating that parking is cheap. Baseline 3 is appropriate since the schema carries the full burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Park until the human presses Send in the review panel, then return that batch.' It clearly states the tool's core function and distinguishes it from siblings by calling it 'the main hand-off' and 'the ONLY way a send reaches you on its own.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-use guidance is provided: 'Call this immediately after annotate_open and again after handling each batch.' It also explains why this tool is necessary (MCP server cannot start a turn) and why alternatives won't work, giving clear contextual exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.