Server Quality Checklist

Tool Scores

• deep_researchA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It discloses key behaviors: query decomposition, real-time web search, code execution when include_analysis=True, synthesis into reports with citations, and cost monitoring. It does not cover all traits (e.g., rate limits, auth), but the main behaviors are well communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (What it does, Best for, Returns, Note). Every sentence adds value, and the key information is front-loaded. It is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 params, output schema exists), the description covers behavior and return type reasonably but lacks parameter explanations for most fields. It omits details on async handling despite a callback_url. Combined with good output schema, it is partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain all 5 parameters. Only include_analysis is partially described (when include_analysis=True). The other parameters (system_instructions, request_clarification, callback_url) are not explained, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs autonomous deep research with web search and analysis, using a strong verb and resource. It does not explicitly differentiate from siblings like research_status or research_with_context, but the scope is clear enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Best for' section lists ideal use cases (current events, data analysis, complex topics, academic research), providing clear context. However, no exclusions or direct comparisons to sibling tools are given, limiting guidance on when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• research_with_contextA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It explains the process (answers to enriched query to research) and return type, but omits behavioral traits like whether the tool is synchronous/asynchronous, potential side effects, or state changes. The callback_url parameter hints at async but is not mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is compact, uses headings and bullet points, and is front-loaded with the main purpose. Every sentence adds value, though it could be slightly more concise by merging lines. No superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the return description is adequate. However, the tool is part of a multi-step workflow (deep_research → research_with_context → research_status), and the description does not explain that results may be polled via research_status. The callback_url parameter hints at async but is not contextualized.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% from description. Description only indirectly explains 'session_id' and 'answers' via usage context, but does not describe 'system_instructions', 'include_analysis', or 'callback_url'. The two required params are partially clarified, but the three optional ones are ignored.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Perform research using an enriched query based on clarification answers' and differentiates from siblings by associating with deep_research's clarification flow. The tool's role in the workflow is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies 'Use after: Calling deep_research with request_clarification=True... Receiving clarifying questions... Gathering answers from user.' This provides clear when-to-use and when-not-to-use guidance relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• research_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return format (status and timestamps) and implies a non-destructive read operation. Adequate for a simple check tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, uses headings and bullet points, and every sentence is useful. It is front-loaded with the purpose and efficiently conveys usage and return info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (one param, simple return), the description is sufficient. It covers the key behavioral aspects and return values, though it leaves out error handling and edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for the only parameter, task_id. The description does not add any additional meaning, such as expected format or origin of the task_id, which is necessary given the lack of schema docs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Check the current status and progress of a running research task,' using specific verb and resource. It effectively distinguishes from siblings like deep_research by focusing on status monitoring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when:' scenarios including long-running tasks and completion checks. While it doesn't list alternatives or when not to use, the context is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.