Server Quality Checklist

Tool Scores

• perplexity_mediumA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's reasoning approach ('enhanced reasoning'), search characteristics ('moderate search depth'), and mentions it returns citations. However, it doesn't disclose important behavioral aspects like rate limits, authentication requirements, error conditions, or what happens with the optional messages parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and efficiently organized with clear sections: purpose statement, 'Best for' guidelines, and parameter explanations. Every sentence adds value, and the information is front-loaded with the most important details first. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (returns dictionary with content and citations), the description doesn't need to explain return values in detail. It covers the tool's purpose, usage guidelines, and parameter semantics adequately. However, for a reasoning/search tool with no annotations, it could provide more behavioral context about limitations, performance characteristics, or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the lack of parameter documentation in the schema. It provides clear explanations for both parameters: 'query: The question or prompt to send to Perplexity' and 'messages: Optional conversation context (list of {"role": "user/assistant", "content": "..."})'. This adds substantial value beyond the bare schema, though it could provide more detail about message format expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'enhanced reasoning with moderate search depth using sonar-reasoning-pro' and specifies it's for 'complex questions requiring analysis, moderate research depth, technical explanations with citations.' This provides a specific verb ('reasoning') and resource ('search'), though it doesn't explicitly differentiate from siblings beyond mentioning 'moderate' depth.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'Best for:' section that lists specific use cases (complex questions requiring analysis, moderate research depth, technical explanations with citations). While it doesn't explicitly say when NOT to use it or name alternatives, the context of having sibling tools (perplexity_large, perplexity_small) combined with the 'moderate' qualifier provides clear guidance on when this specific tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• perplexity_smallA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses behavioral traits like 'quick and reliable,' 'fast factual questions,' and 'optimal speed and cost-effectiveness,' which adds context about performance and constraints. However, it lacks details on rate limits, error handling, or authentication needs, leaving some gaps in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It front-loads the purpose and usage guidelines, followed by clear sections for Args and Returns. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a query tool with 2 parameters), no annotations, and an output schema exists (indicating returns a dictionary with content and citations), the description is mostly complete. It covers purpose, usage, parameters, and returns, but could benefit from more behavioral details like rate limits or error cases to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning beyond the schema by explaining 'query' as 'The question or prompt to send to Perplexity' and 'messages' as 'Optional conversation context (list of {"role": "user/assistant", "content": "..."})'. This provides clear semantics for both parameters, effectively compensating for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Quick and reliable queries using Perplexity's sonar-pro model.' It specifies the action (queries) and resource (Perplexity's model), but doesn't explicitly differentiate from sibling tools (perplexity_large, perplexity_medium) beyond mentioning 'small' in the name and 'optimal speed and cost-effectiveness.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Best for: Fast factual questions, basic research, immediate answers.' It also mentions 'Uses default parameters for optimal speed and cost-effectiveness,' which helps distinguish it from alternatives. This clearly indicates when to use this tool versus potential siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• perplexity_largeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the tool 'may take significantly longer (potentially 10-30 minutes) and may timeout on very complex queries,' uses 'high reasoning effort and search context size,' and returns 'Dictionary with content and citations.' This covers execution time, resource usage, and output format without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by usage guidelines, behavioral warnings, and parameter explanations. Every sentence adds value: the first defines the tool, the second specifies use cases, the third details behavioral traits, and the last sections document parameters and returns. There is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (deep research with potential timeouts), no annotations, and an output schema present (which handles return values), the description is complete. It covers purpose, usage, behavioral transparency (including critical timeout warnings), and parameter semantics, providing all necessary context for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context for both parameters: 'query: The question or prompt to send to Perplexity' and 'messages: Optional conversation context (list of {"role": "user/assistant", "content": "..."})'. This clarifies the purpose and format of each parameter beyond the bare schema, though it doesn't provide exhaustive details like message structure constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'Comprehensive research with maximum depth using sonar-deep-research' and specifies it's for 'Deep research tasks, comprehensive analysis, complex multi-step reasoning, academic research, detailed technical investigations.' This provides a specific verb (research) with clear scope and distinguishes it from sibling tools (perplexity_medium, perplexity_small) by emphasizing maximum depth and comprehensive analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Best for: Deep research tasks, comprehensive analysis, complex multi-step reasoning, academic research, detailed technical investigations' and includes a WARNING about longer execution times. This provides clear guidance on when to use this tool versus alternatives (implied to be the other perplexity tools for less intensive tasks) and when not to use it (time-sensitive queries).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.