Server Quality Checklist

Tool Scores

• tavily_crawlA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, and the description adds no extra behavioral context such as crawl limits, latency, robots.txt handling, or result pagination. The mention of 'depth and breadth' simply reflects parameter semantics, not behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—two short sentences that immediately convey the tool's core function. No redundant words or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 11 parameters and a related output schema, the description offers only a high-level overview. It does not discuss configuration scenarios, limitations, or typical use cases. The thorough schema and output schema partially compensate, but the description alone is insufficient for an agent to understand the tool's full range and trade-offs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptive parameter definitions (e.g., limit, format, max_depth). The description adds no additional meaning beyond what the schema provides, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Crawl a website starting from a URL') with a clear resource and scope. It also mentions extract content and configurable depth/breadth, distinguishing it from sibling tools like tavily_search and tavily_extract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for full-site crawling from a root URL, but does not explicitly state when to prefer it over alternatives such as tavily_extract or tavily_search. No exclusions or 'when-not-to-use' guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tavily_extractA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the tool's non-destructive nature is known. The description adds the output format (markdown/text) but does not disclose potential failure modes, rate limits, or handling of protected pages. It adds modest context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler or redundancy, front-loading the primary action ('Extract content from URLs') before stating the return format. It is highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich input schema (full parameter descriptions), output schema, and safety annotations, the description is largely sufficient. However, it lacks usage differentiation from sibling tools and omits any mention of when advanced extraction depth is needed, though those are partially covered by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with each parameter described, including enums for format and extract_depth. The description adds no additional semantics beyond the schema, so it meets the baseline but does not elevate it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Extract content from URLs' and specifies the return format (markdown or text). This distinguishes it from siblings like search (which queries) and crawl (which follows links), focusing on explicit URL inputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use extract versus alternatives like tavily_crawl or tavily_search. It does not mention exclusion criteria or the specific use case of extracting from known URLs, leaving the agent to infer appropriate usage from the tool name and schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tavily_mapA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read operation. The description adds the return type (list of URLs) and the start point (base URL). It does not add context about external network access, rate limits, or how filters like select_paths affect behavior, so with annotations covering safety, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose. Every word earns its place, and it avoids unnecessary detail or redundancy. This is exemplary conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema and comprehensive parameter documentation, the description does not need to explain return values or parameter syntax. It succinctly conveys the tool's high-level function. However, for a tool with 8 parameters and external web interaction, a bit more context about crawling behavior (e.g., depth/breadth limits) would improve completeness, though it's already adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with detailed parameter descriptions for all 8 fields. The description itself does not mention any parameters or add meaning beyond the schema, so the baseline of 3 is correct.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Map a website's structure' and specifies the output: 'Returns a list of URLs found starting from the base URL.' This is a specific verb+resource+outcome. However, it does not explicitly differentiate from sibling tool tavily_crawl, which likely has overlapping functionality, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (for mapping a website's structure) but provides no explicit guidance on when to prefer this over siblings like tavily_search, tavily_extract, or tavily_crawl. There are no alternatives or exclusion criteria mentioned, so the guidance is only implied by the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tavily_researchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint. The description adds useful behavioral context: a rate limit ('20 requests per minute') and the types of sources included ('web pages, documents, and other resources'). It does not contradict annotations and gives extra value beyond them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Purpose is front-loaded and usage guidance follows. The rate limit sentence is useful. The return-value sentence is somewhat redundant with the output schema, but the overall length is reasonable at four sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only research tool, the combination of schema, annotations, and output schema covers most aspects. The description adds usage guidance, rate limit, and source scope. It lacks an explicit comparison to sibling tools, but the core context is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%; both 'input' and 'model' are already described in detail in the schema. The description does not add additional parameter-level meaning, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Perform comprehensive research') and target ('given topic or question'), and distinguishes itself from siblings by emphasizing 'multiple sources' and comprehensive synthesis. This differentiates it from tavily_search, tavily_extract, tavily_crawl, and tavily_map.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this tool when you need to gather information from multiple sources... to answer a question or complete a task.' This gives a clear usage context, but it does not explicitly name alternatives or state when not to use it. The 'multiple sources' hint implies a distinction from single-source tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tavily_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, and non-destructive behavior. The description adds the return format ('Returns snippets and source URLs'), which is useful behavioral context beyond annotations. It doesn't mention rate limits or pagination, but given the annotation coverage, this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences that are front-loaded with the core purpose. Key usage guidance and return format are included without unnecessary verbosity. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a rich output schema and 100% parameter documentation, the description doesn't need to explain return values or parameters. It provides the essential purpose, usage context, and behavioral output format, making it complete for tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all 15 parameters are documented in the schema. The description itself adds no parameter-level meaning beyond what's already in the input schema. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search the web for current information on any topic' with a specific verb and resource. It also mentions usage for 'news, facts, or data beyond your knowledge cutoff,' which gives context but doesn't explicitly differentiate from sibling tools like tavily_extract or tavily_research.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use this tool: 'Use for news, facts, or data beyond your knowledge cutoff.' It doesn't specify exclusions or alternatives, but the context is clear enough for an agent to understand when web search is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.