Server Quality Checklist

Tool Scores

• quick_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the search is 'quick' and returns 'raw' results, which adds some behavioral context beyond the basic function. However, it lacks details on rate limits, authentication needs, error handling, or what 'raw' specifically entails (e.g., format, source limitations).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose and key behavioral trait ('without deep analysis'), and the Args section efficiently documents parameters. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but with an output schema), the description is fairly complete. It covers purpose, basic behavior, and parameters. Since an output schema exists, it doesn't need to explain return values, but it could benefit from more behavioral details (e.g., speed, source reliability) to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context for both parameters: 'query' is explained as 'What to search for', and 'max_results' includes a default value (5) not explicitly stated in the schema. This goes beyond the schema's basic titles, though it could provide more detail on constraints (e.g., query length, max_results range).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a 'quick web search' and 'returns raw search results', which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'research' or 'analyze' that might also involve searching, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the phrase 'without deep analysis', suggesting this is for basic searches rather than comprehensive research. However, it doesn't provide explicit guidance on when to use this versus alternatives like 'research' or 'analyze', nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gather_sourcesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of behavioral disclosure. It states the tool is keyless, performs search/scrape/dedup, returns full-text sources without writing an answer, and provides numbered blocks. It does not explicitly state it is read-only or non-destructive, but the 'retrieval' nature implies safety. Some details like error handling or rate limits are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a bold lead sentence, usage instructions, and a clear parameter list. It is slightly verbose but each sentence provides value. The front-loading of the key concept ('keyless web retrieval') is effective. The length is appropriate for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and an output schema (not shown), the description covers the main purpose, parameters, and usage workflow. It lacks details on error handling, empty results, or performance characteristics. However, the output schema likely covers return value format, so the description is moderately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description compensates fully. It explains each parameter: query (one focused query, issue multiple for multi-part), max_sources (default 10, bounded 1-20), chars_per_source (default 7000, bounded 500-10000), ranker (lexical default, flashrank optional, or none), and render_thin_pages (sends thin-page URLs to Jina Reader). These details add significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'keyless web retrieval: search + scrape + dedup, returning the top FULL-TEXT sources for a query WITHOUT writing an answer.' It explains the workflow for research. However, it does not explicitly differentiate from sibling tools like quick_search, gather_bundle, or read_url, missing an opportunity to clarify when to use this tool over others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this to research a question: call it several times with different focused sub-queries, read the numbered [Source N] blocks, cross-reference them, then write the answer yourself.' It also advises what to do if sources lack an answer. However, it does not contrast with alternative tools or specify when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_urlA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries behavioral disclosure. It explains that it fetches the page, strips navigation/scripts/ads, returns up to 8000 characters, handles retries, anti-bot protection, and Google Cache fallback. This is comprehensive for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (5 sentences) with front-loaded purpose. Each sentence serves a purpose: action, use case, output specifics, handling mechanisms, and parameter details. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one parameter and an output schema. The description explains the output (title, URL, clean text length) and error handling (retries, cache fallback). It is mostly complete, though it could mention error responses for unreachable pages.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage for 'url', but the description adds an example and the requirement for a full URL (e.g., including protocol). This provides needed context beyond the schema's type definition, though more details on validation could improve.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a URL and extracts clean text. It specifies the verb 'Read and extract' and resource 'clean text content from a URL'. It also mentions it's useful before or after research(), distinguishing it from sibling tools like gather_sources which likely handle multiple sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Useful for reading a specific source in detail before or after running research()', providing clear context when to use. It does not explicitly state when not to use, but the context sufficiently guides an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• gather_bundleA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It thoroughly discloses behaviors: no answer synthesis, default ranker, fallback to lexical_v1, source quality remaining null, and diagnostic action. It also explains parameter bounds and defaults. This provides complete behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a single-line summary, a compact behavior paragraph, and a bulleted Args list. Every sentence adds value without redundancy. The structure is front-loaded with the core action, then details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, output schema), the description covers essential context like return type (SourceBundle), lack of synthesis, and diagnostic guidance. It does not explain what a 'keyless SourceBundle' is or how diagnostics work, which could be clarified, but overall it provides sufficient context for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates fully. Each of the 5 parameters is described with purpose, default values, and bounds (e.g., 'max_sources' bounded to 1-20, 'ranker' options explained). This adds significant meaning beyond the schema's titles and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'structured, keyless SourceBundle without synthesizing an answer,' with specific verb and resource. It distinguishes itself from 'gather_sources' by being 'programmatic' and 'keyless,' and from siblings like 'quick_search' by emphasizing structured evidence identifiers and provenance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the tool is 'intended for agents and pipelines that need stable evidence identifiers and retrieval provenance,' and instructs to follow 'diagnostics.recommended_action' and that only 'synthesize' permits synthesis. However, it does not explicitly compare when to use this versus sibling tools like 'gather_sources' or 'quick_search,' leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.