Server Quality Checklist

Tool Scores

• refresh_selectorsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals key traits: returns the dropped entries, lazy re-derivation requiring a page, and config-pinned selectors are never touched. This is meaningful beyond what a schema would provide, though it doesn't mention irreversibility or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured. The first sentence states the core purpose, followed by additional behavior in short, focused sentences. Every sentence contributes unique information, and the description is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, return value, lazy re-derivation, and an important edge case (config-pinned selectors). However, given the missing parameter semantics and lack of explicit usage guidance, the description isn't fully complete for an agent to use the tool without further assumptions. The presence of an output schema reduces the need to explain return values, but the parameter gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description fails to compensate. It refers to 'these fields' but does not clarify the format or semantics of 'domain' or 'fields', leaving the agent without guidance on what values to provide. The parameters are only given as string types with no further explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Forget the cached selectors for these fields so the next scrape re-derives them.' The verb 'forget' (invalidate) and resource 'cached selectors' are specific, and this distinguishes it from siblings like list_cached_selectors (listing) and scrape (performing the scrape).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when this tool is relevant—when you want to force re-derivation of selectors on the next scrape. It notes that re-derivation is lazy and happens on next scrape, and that config-pinned selectors are untouched. However, it doesn't explicitly contrast with alternatives or state when not to use it, so it stays one point below the top.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_cached_selectorsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses a key behavioral trait (costs nothing) and is clearly read-only via 'Show,' but it doesn't explain what happens when no selectors exist or if authentication is needed. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence with an added cost hint, both earning their place. It's front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with an output schema and a single parameter, the description covers the core purpose and cost behavior. It doesn't need to detail return values due to the output schema. Minor gap: no mention of empty results or preconditions, but acceptable for simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for the 'domain' parameter. The description only repeats 'domain' without providing format, examples, or constraints, so it adds little meaning beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows CSS selectors already learned for a domain, using a specific verb ('Show') and resource. It distinguishes from siblings like scrape and refresh_selectors by focusing on cached selector lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes that the operation 'costs nothing,' implying it's a low-cost read operation. While it doesn't explicitly name alternatives, this context suggests it can be used to check cache before more expensive operations. Could be more explicit, hence 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepare_issue_reportA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states that the tool does not submit the report, checks for duplicates, stops if duplicate is true, and relies on the user to open submit_url. It also warns against including credentials, private URLs, or personal data, adding important context about what the tool does and does not do.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose and the critical non-submitting behavior. Each sentence earns its place by explaining the workflow, duplicate handling, authorization, or privacy constraints, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a complete overview of the tool's workflow, including duplicate detection, user authorization, and the fact that the user manually opens submit_url. Since an output schema exists, it needn't detail return values, but the mention of `duplicate`, `body`, and `submit_url` gives a good sense of expected output. The only gap is parameter semantics, which is covered in that dimension.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description never mentions the four required parameters (summary, how_to_reproduce, cause, suggested_fix). Although the parameter titles are self-explanatory, the description adds no parameter-level guidance or context, failing to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool prepares a bug report about reapfield itself and explicitly highlights that it does NOT submit it. The verb 'prepare' plus the scope 'bug report about reapfield' distinguishes it from sibling tools that manage selectors, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Call this only when you hit a real bug in reapfield during real work and have verified a fix.' It also outlines the workflow (search existing issues, stop if duplicate, show body and ask for authorization), which clarifies the intended usage and how it differs from a submission tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapeA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explains that missing fields yield null with a `misses` entry, that this is normal, and clarifies cost implications of `scroll`, `paginate`, and `no_llm`. It even preemptively notes that `strict` does not exist, which prevents confusion. This is thorough and transparent for a scraping tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet information-dense. It is structured into clear sections: purpose, field syntax, expected missing-field behavior, cost control, and a note about `strict`. Every sentence adds value with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (10 params, no annotations, output schema present), the description covers core behavioral aspects such as output for missing fields, cost implications, and caching hints. It doesn't explain every parameter (mode, refresh, no_cache, cache_ttl) or discuss auth/rate limits, but the output schema and naming conventions cover some gaps. Overall, it is fairly comprehensive but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% for the 10 parameters, so the description must compensate. It does provide detailed meaning for `fields` (comma-separated inline types), `no_llm`, `max_llm_calls`, `scroll`, and `paginate`. However, `mode`, `refresh`, `no_cache`, and `cache_ttl` are left unexplained, relying on their names for inference. The description adds significant value but doesn't fully cover all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Extract structured fields from a web page.' This clearly states what the tool does and distinguishes it from sibling tools like list_cached_selectors or refresh_selectors, which manage selectors rather than perform extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on how to use the tool, including the `fields` syntax and cost-control options (e.g., no_llm, max_llm_calls, scroll/paginate costs). It does not explicitly mention when not to use it or name alternative tools, but the provided context is enough for an agent to decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.