Server Quality Checklist

Tool Scores

• researchA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden and does so richly. It explains the iterative loop, polling behavior with the exact `{"state": "researching"}` reply, billing semantics ('Billed as ONE research run... ONLY when it completes; polling is free'), grounding constraints ('written ONLY from those passages - never invented'), and that the answer names coverage gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but information-dense, with the core purpose front-loaded and alternative tools explicitly excluded. Each sentence adds meaningful operational or selection detail, though the length could be trimmed slightly without losing critical content; it does not waste words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex, multi-phase research tool with no annotations and a high-stakes polling/billing model. The description covers the full workflow, return shape ('cited evidence pack (`passages` with source urls)' and 'grounded `answer`'), polling contract, billing rules, and honesty about unsupported conclusions. The presence of an output schema means return-values don't need full enumeration, but the description goes well beyond minimum.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, so the description must compensate. It provides strong semantics for the query parameter (the kind of question to investigate) and indirectly covers include_answer by mentioning 'by default, a grounded answer.' However, max_tokens is never explained in the description, and the schema itself only provides a title and default, leaving this parameter's purpose ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a strong verb and resource: 'INVESTIGATE an open question that has no ready-made answer sitting on any single page.' It clearly distinguishes itself from siblings by contrasting with deep_search, web_search, and fetch, emphasizing that it synthesizes across multiple sources rather than retrieving one result.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance with concrete examples (open/analytical, contested, multi-part) and explicit when-not-to-use instructions ('Do NOT use it for a quick fact (web_search), a known page (fetch), or a question one search settles (deep_search)'). It also mentions cost and latency trade-offs, helping the agent select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_searchA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses cost ('Costs 1 page'), result count ('Returns up to 10 results'), and a key behavioral nuance (snippets often contain the answer). This goes above and beyond typical descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense yet concise. Every sentence contributes value: it lists return fields, surfaces, usage guidance, result limits, and cost. The structure flows logically from core function to operational detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with an output schema, the description is highly complete. It covers what is returned, the surrounding surfaces, when to use alternatives, result caps, and cost. The only gap is the region parameter, but that is a minor omission given the overall richness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for parameter meaning. The query parameter is implied but never explicitly described, and the region parameter is completely absent. This leaves the agent guessing about the region format and effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Google and returns organic results with specific fields (title, URL, snippet, etc.), and distinguishes itself from siblings by mentioning fetch and deep_search. The verb-resource combination is precise and the scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly guides when to use this tool vs alternatives: 'Use `fetch` when you need the full page, and `deep_search` when one page will not settle it.' Also advises reading snippets before fetching, providing clear decision rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deep_searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden and excels: it discloses that the tool returns evidence, not a written answer; that the agent must compose and judge the answer; that the 'include_answer' field is optional and omitted when unsupported; and that the pack is capped to fit context and costs per page read.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat lengthy but well-structured: it front-loads the core purpose, then covers usage, behavior, and parameters. Each sentence adds value, though it could be tightened by merging a few related points without losing substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is remarkably complete for a complex tool: it covers purpose, usage, alternatives, return behavior, parameter semantics, costs, performance, and limitations. The output schema is mentioned ('Returns `passages`') and the return values are explained, so the agent has all the information needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the description adds meaning for the non-obvious parameters: it explains max_tokens ('pack is capped to fit your context window'), max_fetches ('Costs 1 page per page read (default up to 6)'), and include_answer ('set include_answer: true and the reply ALSO carries answer'). The required query parameter is self-evident.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search the web across several pages to answer a question in ONE call' and details the multi-step process (search, fetch, rank, return verbatim slices). It explicitly distinguishes itself from siblings by naming web_search and fetch with contrasting use cases, making it unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use and when-not-to-use guidance with concrete examples: 'Use this for a question that needs reading and comparing MULTIPLE pages' and 'Do NOT use it for a quick fact a single search snippet already answers (use web_search), or to read ONE page whose URL you already have (use fetch).' This is exemplary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: synchronous call, no polling, failure reported honestly rather than fabricating content, bot-walls and consent screens reported as failures, and the cost model (1 page per success, free on failure). This is unusually transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action, then flows into usage guidance, parameter clarification, and failure semantics. Every sentence adds operational value and there is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description adds essential context: return fields (content, title, description, final url), passage retrieval via query, grounding constraints, failure behavior, and cost. This is complete for a tool with this complexity and no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for url (grounded sources), clean (false returns raw HTML), and query (returns relevant passages). However, max_tokens is never mentioned, leaving one parameter unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetch the full, clean content of ONE web page as markdown.' It clearly frames this as reading a search result when the snippet is insufficient, which distinguishes it from siblings like web_search and deep_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: check the search snippet first, use fetch when the snippet isn't enough, and the URL must come from a recent search in the session. It also explicitly states what the tool is not ('not a general-purpose scraper'), giving a clear boundary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.