Server Quality Checklist

Tool Scores

• deep_researchB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool uses 'Perplexity's Sonar Deep Research model' and is for 'comprehensive research topics,' but it lacks details on behavioral traits such as response format, potential rate limits, authentication needs, or whether it's a read-only or mutative operation. The description doesn't contradict annotations, but it's insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of two concise sentences that directly state the tool's purpose and usage context. There's no wasted text, and it efficiently conveys key information without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no annotations, no output schema), the description is moderately complete. It covers the purpose and hints at usage but lacks details on behavioral traits, output format, and differentiation from siblings. The high schema coverage helps, but for a research tool with no output schema, more context on what to expect from results would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the input schema already documents all parameters thoroughly. The description adds no additional meaning beyond what the schema provides—it doesn't explain parameter interactions or usage nuances. With high schema coverage, the baseline score is 3, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Conducts in-depth analysis and generates detailed reports using Perplexity's Sonar Deep Research model.' It specifies the verb ('conducts analysis and generates reports'), resource ('Perplexity's Sonar Deep Research model'), and scope ('comprehensive research topics'). However, it doesn't explicitly differentiate from sibling tools 'reason' and 'search' beyond mentioning it's 'best for comprehensive research topics,' which is somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some guidance: 'Best for comprehensive research topics.' This implies usage for complex or broad inquiries, but it doesn't explicitly state when to use this tool versus alternatives like 'reason' or 'search,' nor does it mention any exclusions or prerequisites. The input schema hints at usage context by emphasizing specificity, but this isn't part of the description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the model (Sonar Pro) and that it's for 'quick search,' but lacks details on rate limits, authentication needs, response format, or error handling. For a search tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and usage guidelines. Every word earns its place, with no redundancy or fluff. It's appropriately sized for a simple search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers purpose and usage but lacks behavioral details (e.g., response format, limitations). Without annotations or output schema, the description should do more to compensate, but it's minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the two parameters. The description doesn't add any parameter-specific information beyond what's in the schema. According to the rules, with high schema coverage (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Quick search for simple queries using Perplexity's Sonar Pro model.' It specifies the action (search), the resource (information via Sonar Pro model), and the scope (simple queries, basic information lookup). However, it doesn't explicitly differentiate from its siblings 'deep_research' and 'reason' beyond implying simplicity vs. complexity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'Best for straightforward questions and basic information lookup.' This implies when to use it (simple queries) and when not to use it (complex queries, which might be handled by siblings like 'deep_research' or 'reason'). However, it doesn't explicitly name alternatives or state exclusions, keeping it at a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reasonA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions using 'Perplexity's Sonar Reasoning Pro model' and hints at complexity handling, but lacks details on performance traits (e.g., latency, rate limits), error handling, or output format. It adds some context but falls short of fully describing behavioral aspects for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of two sentences that directly state the tool's purpose and best-use cases. Every sentence earns its place by providing essential information without waste, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description is incomplete. It covers purpose and usage well but lacks details on behavioral traits, output format, or error handling. For a tool with 2 parameters and no structured metadata, the description should do more to compensate, leaving gaps in contextual understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description does not add specific parameter semantics beyond what's in the schema, but since coverage is high, the baseline is 3. It earns a 4 because the description implicitly reinforces the importance of the 'query' parameter by emphasizing complex tasks, adding slight contextual value without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'handles complex, multi-step tasks' using a specific reasoning model, with examples of use cases (explanations, comparisons, problem-solving). It distinguishes from 'deep_research' and 'search' by emphasizing reasoning over research or simple search, though not explicitly naming alternatives. The purpose is specific but could be more explicit about sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Best for explanations, comparisons, and problem-solving'), implying it's suited for complex reasoning tasks. However, it does not explicitly state when not to use it or name alternatives like 'deep_research' or 'search', missing explicit exclusions or comparisons. The guidance is strong but not fully comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.