Server Quality Checklist

Tool Scores

• get_locatorB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are available, so the description must fully disclose behavior. It only states that a single locator entry is returned, but omits important details like error handling for missing keys, side effects (none expected, not stated), or response format. This is insufficient for a mutation-free lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no extraneous content. It is concise and front-loaded, conveying the essential purpose immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 required parameters, no output schema, no annotations), the description covers the basic operation but lacks details on return structure and error behavior. For a simple lookup tool, it is minimally complete but could benefit from mentioning that the locator entry is returned as an object with specific fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning beyond the schema; it merely restates that the tool operates by key and registry name. The parameter descriptions in the schema already clarify the parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a single element locator entry by key from a named registry. It uses a specific verb ('Return') and resource ('single element locator entry'), and distinguishes from siblings like 'get_registry' (returns entire registry) and 'get_registry_keys' (returns keys only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives such as 'search_registry' or 'get_registry'. It lacks explicit usage context, prerequisites, or exclusions, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_registriesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full burden. It only states the operation (list) and location, but does not disclose side effects, permissions needed, output format, or limitations. For a read operation, minimal transparency is given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded and directly states the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema, no annotations), the description is mostly complete. It specifies what is listed and where. However, it could mention whether the output is just names or full paths.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already covers 100%. Per guidelines, baseline is 4 since the description need not add parameter semantics. No parameters to elaborate on.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists all available registry files in a specific directory. The verb 'list' and resource 'registry files' are specific. It implicitly distinguishes from siblings like 'get_registry' and 'get_registry_keys', but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings (e.g., get_registry, search_registry). The description lacks any context about use cases or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_registry_keysA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It only describes the return content but fails to disclose any behavioral traits such as read-only nature, performance characteristics, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action and keeps all information relevant. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description mentions the fields returned, it lacks details on the structure (e.g., array of objects?), pagination, or error conditions. Given no output schema, more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter scanName, with a clear description. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a lightweight list of element keys with specific properties (tagName, testId, confidence) and contrasts it with the full registry, effectively distinguishing from sibling tools like get_registry.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when a lightweight list is needed instead of the full registry) but does not explicitly exclude use cases or mention alternatives beyond the contrast.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_registryA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states it returns matching entries only, but with no annotations, it lacks disclosure on side effects, authorization needs, rate limits, or behavior when no matches found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each serving a purpose. The first sentence specifies action and parameters, the second clarifies return behavior. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 5 parameters and no output schema, the description is missing details on return format, pagination, error handling, or performance implications. It is minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description lists the filter parameters but does not add new semantics beyond the schema; the required scanName is not mentioned in description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a registry and lists the filter parameters (tagName, testId, text, confidence). It distinguishes from siblings like get_registry (full retrieval) and list_registries (listing all).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (when filtering by tag, testId, text, or confidence) but does not explicitly state when not to use or provide comparison with alternatives like get_registry_keys or get_locator.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_registryA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool returns file contents, implying a read-only operation with no side effects. However, it does not explicitly mention permissions, error handling, or confirm lack of destructive impact, leaving some ambiguity about behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose and includes necessary details about the file path and contents. Every word adds value, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter, the description provides complete context: what the tool does (returns registry contents), how to use it (scanName param with example), and what the return structure is (map of locators). No output schema exists, but the description adequately covers return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds value by providing an example value ('goal-side-panel-locators') and showing the file path pattern ('registry/{scanName}.json'), which clarifies the parameter usage beyond the schema's description of 'Registry filename without .json'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full contents of a registry JSON file, specifying the file path pattern and that it contains a map of element keys to CSS, XPath, and template locators. This clearly distinguishes it from sibling tools like get_locator (single locator) or list_registries (listing names).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving the entire registry, but it does not explicitly state when to use it versus alternatives (e.g., get_locator for a single locator, get_registry_keys for keys only). The usage context is implied rather than explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_pageA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains connection modes and registry file naming, but doesn't mention side effects or safety. Since it's a read-like operation, it's acceptable but could note if it alters the page.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with numbered modes, but slightly verbose; could be trimmed without losing clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and sibling tools, the description fully explains operation, parameters, and usage context, leaving no key gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds value by clarifying mutual exclusivity of cdpEndpoint/wsEndpoint and specifying scanName usage, going beyond schema info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans an open browser page to extract element locators, with two specific connection modes, making the purpose unambiguous and distinct from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use each connection mode (cdpEndpoint vs wsEndpoint) with examples, and states exactly one must be provided, providing clear guidance over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.