Server Quality Checklist

Tool Scores

• ImageSearchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses backend behavior (Brave vs DuckDuckGo) and safesearch parameter effects. With no annotations, provides moderate transparency but lacks details on rate limits, permissions, or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise at 3-4 sentences, well-structured, front-loaded with main action. Slightly verbose on backend details but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers return data, optional download, and safesearch. Lacks output schema, but description sufficiently explains returned fields. Completeness is good for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning beyond schema: explains backends, safesearch options in context, and clarifies fetch_top_n downloads locally. Schema coverage is 100%, so baseline is 3; description provides extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Search for images' and details return types (URLs, thumbnails, dimensions) and optional local download. Distinct from sibling tools (PaperFetch, WebFetch, etc.) which are not image-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. Does not mention when not to use or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• WebSearchA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the unfiltered nature of results and the backend API (Brave or DuckDuckGo), which adds value beyond the schema. However, it lacks details on rate limits, result count, error handling, or any potential restrictions (e.g., content blocking by default).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the core purpose, and includes relevant behavioral details without redundancy. Every sentence serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description adequately explains what is returned (title, URL, snippet). However, it omits details on pagination, result count, and error behavior, which could be important for an agent using this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds minimal extra meaning. It mentions 'no domain blocks' which aligns with the allowed/blocked domains parameters, but it does not elaborate on parameter usage or format beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function (search the web for information) and specifies what it returns (title, URL, snippet). It also distinguishes from sibling tools like ImageSearch and PaperFetch by focusing on general web search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (for general web search) but does not provide explicit guidance on when not to use or how it compares to siblings like WebResearch or WebFetch. No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• PaperFetchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It notes extraction is conditional ('when possible') and lists return values, but does not mention authentication needs, rate limits, or potential failure modes for non-open-access papers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences cover all essential information with no wasted words. The first sentence front-loads the action and resource, while the second lists outputs concisely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and no output schema, the description covers purpose, usage, and returns. It lacks error handling details but is otherwise sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents parameters. The description adds minor clarification (URL can be landing page or PDF) but does not significantly augment schema information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads open-access paper PDFs and extracts text. It specifies the resource (arXiv/PDF paper sources) and distinguishes from WebFetch, making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states to use this for arXiv/PDF paper sources before quoting a paper, providing clear context. However, it does not include when not to use it or alternative tools, which would strengthen guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• WebResearchA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states the combined functionality and return type but does not cover failure modes, rate limits, or how parameters interact (e.g., if fetch_top_n exceeds count). Without annotations, more behavioral detail would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with no redundant information. The first sentence captures the core functionality, the second clarifies the output. Ideal length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the high-level behavior but does not specify the output format or how parameters interact. For a research tool combining two operations, users might want to know if the extracted content is plain text, if there are limits on total characters, etc. The description is sufficient for basic understanding but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the schema describes each parameter individually, the tool description explains the interaction: it searches and then fetches the top N (fetch_top_n) of the search results. This clarifies the workflow and the relationship between count and fetch_top_n.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs a combined search and fetch operation, returning both search results and extracted content. This differentiates it from sibling tools like WebSearch (search only) and WebFetch (fetch only), as well as ImageSearch and PaperFetch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description recommends using this tool for research queries, implying it's best when both search and content extraction are needed. It doesn't explicitly exclude other use cases or mention alternatives, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• WebFetchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behaviors: returns raw extracted content, no summarization, automatic handling for Reddit URLs and images, and caching for 24 hours. It does not cover error handling or rate limits, but covers the most critical aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at four sentences, front-loaded with the main purpose, and every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description covers the essential aspects: what is returned, special cases, and caching behavior. It lacks details on error handling and output format structure, but is largely complete for a fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with clear descriptions for both parameters. The description adds extra context about the 'prompt' parameter being ignored and special URL handling, but no additional detail beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('fetch') and resource ('content from a URL'), lists specific return fields (title, headings, text, links), and distinguishes it from sibling tools like ImageSearch and WebSearch which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (when you need raw content from a URL) and what it does not do (no summarization). It also covers special cases like Reddit and images. However, it does not explicitly mention when not to use it or provide direct comparisons to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.