Server Quality Checklist

Tool Scores

• perplexity_askB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true and openWorldHint=true, which the description doesn't contradict. The description adds that it 'engages in a conversation' and uses the 'Sonar API', providing some context beyond annotations, but lacks details on rate limits, authentication needs, or specific behavioral traits like response format or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of two sentences that directly state the tool's action and parameters without unnecessary details. Every sentence contributes essential information, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a conversational AI tool with one parameter), the description covers the basic purpose and input. With annotations providing safety hints and an output schema presumably detailing the response, the description is reasonably complete, though it could benefit from more behavioral context or sibling differentiation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'messages' fully documented in the schema. The description mentions 'accepts an array of messages (each with a role and content)', which aligns with but doesn't add meaningful semantics beyond the schema, such as usage examples or constraints on message structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'engages in a conversation using the Sonar API' and 'returns a chat completion response from the Perplexity model', which specifies the verb (engages/returns) and resource (conversation/response). However, it doesn't explicitly differentiate from sibling tools like perplexity_reason or perplexity_search, which likely have similar conversational purposes but different scopes or behaviors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (perplexity_reason, perplexity_research, perplexity_search). It mentions the general action but offers no context about appropriate scenarios, exclusions, or alternatives, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• perplexity_reasonB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true and openWorldHint=true, which the description doesn't contradict. The description adds value by specifying the model (sonar-reasoning-pro) and the purpose (reasoning tasks), but it lacks details on behavioral traits like rate limits, error handling, or response format beyond what annotations provide. No contradiction is present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of two sentences that efficiently convey the core functionality and model used. Every sentence adds value without redundancy, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (reasoning tasks with an API), annotations cover safety (readOnlyHint) and scope (openWorldHint), and an output schema exists, the description is reasonably complete. It specifies the model and purpose, but could improve by differentiating from siblings or adding more context on use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the parameters (messages array and strip_thinking boolean). The description adds no additional meaning beyond what's in the schema, such as examples or usage tips for parameters. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'performs reasoning tasks using the Perplexity API' and 'returns a well-reasoned response using the sonar-reasoning-pro model,' which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like perplexity_ask, perplexity_research, or perplexity_search, leaving some ambiguity about when to choose this tool over others for reasoning tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (perplexity_ask, perplexity_research, perplexity_search). It mentions the model (sonar-reasoning-pro) but doesn't specify use cases, exclusions, or alternatives, leaving the agent without clear context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• perplexity_researchB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only and open-world hints, which the description doesn't contradict. It adds value by specifying that it 'returns a comprehensive research response with citations', providing context on output behavior. However, it lacks details on rate limits, authentication needs, or response format beyond citations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, uses two concise sentences with zero waste, and efficiently conveys key information without redundancy. Every sentence earns its place by adding distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of annotations and an output schema, the description is reasonably complete for a research tool. It covers the basic action and output type, though it could benefit from more context on when to use versus siblings or behavioral traits like response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents parameters. The description adds minimal semantics by mentioning 'array of messages' and 'comprehensive research response', but doesn't elaborate on parameter usage beyond what's in the schema. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Performs deep research') and resource ('using the Perplexity API'), and distinguishes from siblings by specifying 'deep research' rather than generic queries. However, it doesn't explicitly contrast with 'perplexity_reason' or 'perplexity_search' to fully differentiate purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'perplexity_ask' or 'perplexity_search'. The description mentions 'deep research' but doesn't clarify scenarios or prerequisites for choosing this over sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• perplexity_searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, indicating safe, external data access. The description adds valuable behavioral context beyond this by specifying the return format ('ranked search results with titles, URLs, snippets, and metadata') and the tool's strength for 'up-to-date' information, which helps the agent understand output structure and timeliness without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core functionality and output, and the second provides usage context. Every phrase adds value without redundancy, making it front-loaded and appropriately sized for a search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, rich annotations (readOnlyHint, openWorldHint), 100% schema coverage, and the presence of an output schema (implied by context signals), the description is complete enough. It covers purpose, output format, and ideal use cases, leaving technical details to structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing full documentation for all 4 parameters. The description doesn't add any parameter-specific details beyond what the schema already covers, such as explaining how 'country' affects results or typical 'query' formats. This meets the baseline of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Performs web search') and resource ('using the Perplexity Search API'), and distinguishes it from siblings by emphasizing web search functionality rather than conversational or analytical approaches. The phrase 'Perfect for finding up-to-date facts, news, or specific information' further clarifies its distinct use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Perfect for finding up-to-date facts, news, or specific information'), which implicitly suggests it's for factual retrieval rather than reasoning or research tasks handled by siblings. However, it doesn't explicitly name alternatives or state when not to use it, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.