Server Quality Checklist

Tool Scores

• extractA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so description carries full burden. Mentions cost (5 credits) and return fields, indicating it's a read operation. Does not detail error cases, rate limits, or behavior on malformed URLs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences plus a return field list. Front-loaded with purpose. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param tool with clear description, it sufficiently covers purpose, usage, and output. Lacks error handling or edge case details, but overall complete for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage 100%, so parameters are already well-described in schema. Description adds credit cost and return fields but no new parameter-level detail beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes fetching a webpage using AI to extract structured data, clearly distinguishing from raw content. Gives examples like pricing, specs, contact info. Specific verb+resource+scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (for structured data rather than raw content) and mentions credit cost. Suggests alternatives implicitly with sibling tools fetch and search. No explicit when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses credit cost, max URLs, failed URL charging, and behavior of 'include_raw_html' (returns null for non-HTML). It does not mention rate limits or robots.txt, but the disclosed info is solid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear paragraphs. It is concise but covers all key points. Slight redundancy like repeating 'max 10 URLs' could be trimmed, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description compensates by listing the return fields (title, url, etc.). It also addresses error handling, credit usage, and format support. For a tool with two parameters, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds value by explaining the purpose of 'include_raw_html' (inspecting embedded elements) and its behavior for non-HTML content. It also clarifies credit usage per URL, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'fetch', resource 'URLs', and output 'clean markdown'. It distinguishes from siblings by contrasting with 'snippet from search' and mentioning support for PDF/DOCX formats, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool ('read articles, documentation, blog posts') and implicitly suggests when not to use it ('not just a snippet'). It also provides constraints like credit cost, max URLs, and failed URL policy. However, it does not directly compare with sibling tools 'extract' or 'research'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• researchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses key behaviors: query decomposition, parallel source reading, synthesis, and credits consumption. Also lists return fields, offering comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose, followed by usage guidance, cost, and return format. Every sentence is informative with no waste. The structure is logical and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description lists all return fields (query, report, sources, sub_queries, credits_used, etc.), providing complete context for what the agent will receive. The process steps are also described, making the tool's behavior fully understood.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions in the schema. The tool description adds the credit cost but does not elaborate on parameter behavior beyond what is in the schema. Baseline 3 is appropriate as it adds marginal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Perform comprehensive research on a topic' and explains the process (decomposes, searches, reads, synthesizes). It distinguishes from 'search' for simple lookups, making the tool's purpose specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly indicates best use ('open-ended or comparative questions') and when not to use ('simple factual lookups, use search instead'). Mentions cost (25 credits) and alternative tool with detail, providing clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully compensates by disclosing credit costs for different search depths and include_answer, comparing costs to research (8 vs 25 credits), and detailing the return structure including 'altered_query'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear paragraphs: purpose, news guidance, cost details, and return fields. Every sentence is informative, front-loaded with purpose, and no superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 8 parameters and no output schema, the description covers purpose, usage guidelines, parameter behavior, costs, and return format comprehensively. It addresses the tool's complexity adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters. The description adds value by explaining credit costs and recommending when to use topic='news' and include_answer, going beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search the web for current information on any topic' and notes it returns extracted page content. It distinguishes from the research sibling by directing open-ended synthesis questions there.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Best for factual lookups, specific questions, or when you need a list of sources' and when not: 'For open-ended questions that need synthesis across many sources, use the research tool instead.' Also provides guidance for news queries with topic='news'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.