Server Quality Checklist

Tool Scores

• web_searchB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint=false, so the safety profile is covered. The description adds provider capabilities and operator support, which is useful context. It does not disclose response format, pagination, rate limits, or pitfalls, and the large_result_mode parameter is not explained in the description, though the schema covers it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads purpose, then adds provider and operator details. Every clause earns its place, with no repetition or irrelevant information. Structure is clean and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 params and no output schema, the description covers provider selection and basic usage, but it does not explain when to choose specific providers (e.g., when to use kagi vs brave), the semantics of large_result_mode, or the expected return format. The schema fills in param meanings, but given the tool's complexity and lack of output schema, more operational context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by characterizing providers (tavily=factual/citations, brave=privacy/operators, etc.) and mentioning query operators, which helps interpret the provider and query params. It does not add syntax or format details for other parameters like limit or exclude_domains, but the schema already describes those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tautological: description restates name/title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear use context ('Use when you need to find web pages...') and differentiates providers by characteristics (tavily factual, brave privacy, etc.). However, it does not explicitly mention when not to use this tool or point to sibling alternatives like ai_search or web_extract, which would be strong guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ai_searchB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds provider-specific details (e.g., 'fast ~900ms answers', 'deep agentic search with sources') which provide some behavioral context, but it does not disclose potential rate limits, response size handling, or other operational behaviors beyond what annotations cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the core purpose, and every sentence adds value. It is a single paragraph with no fluff, and the provider list is informative without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 params, 2 required, no output schema), the description covers the main use case and provider options. It does not explain return format or pagination, but the schema covers parameters and the annotations cover safety. It is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds provider names and their characteristics, which enriches the 'provider' parameter semantics, but it does not add much beyond the schema for 'query', 'limit', or 'large_result_mode'. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tautological: description restates name/title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use when you need synthesized answers rather than raw search results,' which provides clear usage context. It also lists provider options, but does not explicitly mention when not to use it or alternatives beyond the sibling tools, so it falls slightly short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• web_extractA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare read-only, idempotent, and open-world behavior, so the description need not repeat those. It adds behavioral nuance by describing provider modes (e.g., firecrawl's interactive scraping, exa's similar pages) and mentions configurable options like include_raw_contents and large_result_mode. However, it does not elaborate on potential side effects or limitations beyond what the annotations cover, so it falls short of a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct, using two sentences plus a compact list of providers and their capabilities. Every sentence adds value, and there is no redundant or verbose phrasing. The structure is logical and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's core purpose, typical use cases, provider selection, and handling of large results. It does not require an output schema since none is specified, and the annotations provide safety context. Together, these elements give a complete picture for an agent to decide when and how to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides complete descriptions for all six parameters, and the tool description enriches understanding by explaining provider-specific purposes (e.g., kagi for summarization of pages/videos/podcasts) and the meaning of large_result_mode. While the schema is already thorough, the description adds context that clarifies parameter choices, hence above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tautological: description restates name/title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage contexts by listing provider-specific capabilities (e.g., tavily for extraction, kagi for summarization, firecrawl for scraping/mapping). It does not explicitly contrast with the sibling tools (web_search, ai_search), but the focus on URL-based content processing makes the distinction reasonably clear. A more explicit 'use this when you need content from specific URLs, not for searching' would earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.