Server Quality Checklist

Tool Scores

• pplx_deep_research_startA
  3.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation contradiction: The annotations declare readOnlyHint=true, but the description says 'Start a ... job' and 'Returns a job_id', implying a state-changing operation (creating an async job). This directly contradicts the readOnlyHint. The description does add useful timing info (2-20 minutes) and polling instruction, but the contradiction forces a score of 1 per rules.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core action, then usage context, then follow-up and caveats. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (async job, polling, duration, cost), the description covers all essential operational context: immediate return, polling with job_id, typical duration, and cost/overkill warning. It also distinguishes from normal web search. The contradictory annotation is a separate issue, but the description itself is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters. The tool description adds no parameter-level guidance beyond the schema. Per the baseline rule, score is 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Start a Perplexity Sonar Deep Research job' with the specific verb 'start' and resource. It distinguishes from sibling tools by noting 'After calling this, poll pplx_deep_research_check with the returned job_id' and implies listing via 'start' vs 'list'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases: 'exhaustive multi-source investigation: literature reviews, market and competitor analysis, regulatory landscapes.' It also tells when NOT to use it: 'For quick factual lookups this is overkill and expensive - use a normal web search instead.' This is clear guidance with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pplx_deep_research_listA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds that it lists 'recent' jobs, implying a temporal ordering, and that it returns ids, models, and statuses, but it does not disclose additional behavioral traits such as pagination or error conditions. This is consistent with the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first defines the action and output, the second gives practical use cases. It is front-loaded with the verb and resource, with zero filler words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters, and the schema plus annotations already provide the necessary context. The description explicitly lists the returned fields (ids, models, statuses), which compensates for the lack of an output schema, and the use cases make the tool's purpose clear. This is complete for a list-style tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100%, with descriptions for both 'limit' and 'status', so the baseline is 3. The description does not add any extra explanation of the parameters beyond the schema; it only references the returned 'ids' which are not a parameter. Therefore no additional credit is warranted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and identifies the resource ('deep research jobs') plus the returned attributes ('ids, models and statuses'). It also distinguishes itself from siblings by explaining the use case of recovering a lost job_id, which differs from starting or checking specific jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to recover a job_id you lost, or to see what is still running,' providing clear scenarios for when to invoke this tool. It does not name the sibling alternatives or state when not to use it, so it falls short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pplx_deep_research_checkA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, openWorld, and non-destructive behavior. The description adds valuable context beyond annotations: it returns immediately with current status, supports polling, and results remain retrievable for 7 days. This enriches understanding without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four short sentences, each carrying distinct information: purpose, immediate return, polling behavior, and 7-day retention. It is front-loaded with the primary action and contains no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters and no output schema, the description covers the main behavioral aspects: what it does, when to call, and how long results persist. It does not detail the exact return format, but 'full report' and 'current status' give reasonable guidance. With sibling context, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for all three parameters (job_id, wait_seconds, strip_thinking). The tool description itself adds no additional parameter-specific meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: checking a deep research job and retrieving the full report once finished. It uses specific verbs ('check', 'retrieve') and identifies the resource ('deep research job'), distinguishing it from siblings like pplx_deep_research_start (starts jobs) and pplx_deep_research_list (lists jobs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it is for checking a job started via pplx_deep_research_start, and explicitly advises 'If the job is still running, call again.' It also mentions the 7-day retention period. However, it does not explicitly state when not to use it or compare it to list alternatives, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.