Server Quality Checklist

Tool Scores

• find_apisC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only states the purpose. It doesn't mention whether the tool performs live searches, what evaluation criteria are used, what output format is returned, or any side effects. This is a significant lack of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded and directly conveys the tool's purpose. Every word earns its place, and there is no unnecessary detail or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description should compensate by explaining return values, research process, or usage context. It does none of this, leaving the agent without a clear picture of the tool's output, behavior, or when to choose it over siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (requirement and context) with 100% coverage, so the baseline is 3. The tool description adds no additional meaning beyond the schema; it doesn't explain how the parameters influence behavior or results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds and evaluates APIs for project integration, using a specific verb and resource. It distinguishes itself from sibling tools like general search and get_documentation, though it doesn't detail the evaluation criteria or depth.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. It doesn't mention any exclusions, prerequisites, or comparisons with sibling tools like search or get_documentation, leaving the appropriate usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_deprecated_codeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description does not disclose behavioral traits beyond its basic function. It does not state if the check is read-only, whether it executes the code, what input format is expected, or what output is produced. With no annotations, this lack of detail is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no redundant information. It gets straight to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the small parameter count, the description lacks crucial information about the tool's behavior and return value. Without an output schema or annotations, the agent is left without a clear picture of what to expect. The description is too minimal to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes both parameters (code and technology), so the description doesn't need to elaborate. The description adds no additional parameter context beyond the schema, aligning with the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: checking code or dependencies for deprecated features. It distinguishes from sibling tools like search and get_documentation by focusing on deprecation analysis rather than general information retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like search or get_documentation. The description only states what it does, implying usage but offering no exclusions or comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only promises 'comprehensive information' without noting whether the tool is read-only, what data source it accesses, whether it has rate limits, or what the response format looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The one-sentence description is efficient and front-loaded with the verb and resource. It contains no filler, though its brevity limits the amount of actionable guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple schema and lack of an output schema, the description still leaves important gaps: it does not explain what kind of results are returned, when this tool should be preferred over sibling tools, or how the optional detail_level affects behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters with descriptions, giving 100% coverage. The description adds only generic context ('any topic') and does not enrich the meaning of the parameters beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Perform a general search query') and resource ('search') to gather information. However, it does not differentiate this from sibling tools like get_documentation or find_apis, so it stops short of full clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to choose this tool over alternatives. The phrase 'any topic' is extremely broad and gives no context or exclusions, leaving the agent without direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentationB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the primary function and reveals nothing about potential side effects, output format, source reliability, or limitations. The agent is left guessing about what 'documentation' entails (e.g., official docs, community examples, version specifics).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the main verb and resource. There is no wasted wording or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with full schema coverage and no output schema, the description is minimally viable but lacks usage guidance and behavioral context. It could be more complete by explaining what types of documentation are returned or how it differs from sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both parameters, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides; it merely reiterates 'specific technology, library, or API' which matches the 'query' parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and the resource ('documentation and usage examples') for a 'specific technology, library, or API.' This is specific and distinguishes it from generic search, though it doesn't explicitly call out sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like search, find_apis, or chat_perplexity. The description implies a documentation-focused purpose but provides no exclusions, prerequisites, or contextual cues for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chat_perplexityA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does reveal a key behavior: 'full history context' indicates that previous messages are maintained and used. However, it does not disclose other relevant behaviors such as potential errors with invalid chat_id, response format, latency, or any side effects. The description adds some value but remains minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with two short sentences that effectively convey the tool's core functionality. It is front-loaded with the primary purpose and includes only essential additional context about chat continuation. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with two parameters, the description covers the main purpose but omits the return value or behavior details (e.g., what response is returned to the user). There is no output schema to compensate, and no annotations. The description is minimally viable but leaves some uncertainty about the tool's actual output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both 'chat_id' and 'message' already documented in the schema. The description adds little beyond the schema, though it reinforces that chat_id is optional and controls chat creation/continuation. Since the schema already covers semantics, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: maintaining ongoing conversations with Perplexity AI, explicitly covering both new chat creation and continuation of existing ones with full history. It uses specific verbs (maintains, creates, continues) and identifies the resource (conversations with Perplexity AI), distinguishing it from sibling tools that perform search or documentation retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a conversational interaction with Perplexity AI is needed, including creating a new chat or continuing an existing one. However, it does not explicitly compare with alternatives or state when not to use this tool (e.g., for simple search queries). The context is clear but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.