Server Quality Checklist

Tool Scores

• search_docsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that queries should be specific and that omitting 'api' affects results, which adds some context about search behavior. However, it fails to disclose critical traits like whether the search is read-only, if it has rate limits, authentication needs, or what the output format looks like (especially since there's no output schema). This leaves significant gaps for an agent to understand the tool's behavior fully.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, starting with the core purpose in the first sentence. The second sentence provides practical advice without unnecessary elaboration. Both sentences earn their place by adding value, though the structure could be slightly improved by explicitly mentioning parameters or output expectations to enhance clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (semantic search with three parameters) and the absence of both annotations and an output schema, the description is incomplete. It explains the basic purpose and offers usage tips but fails to cover behavioral aspects like safety, performance, or return values. This leaves the agent with insufficient information to use the tool confidently in varied contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting all three parameters (query, service, top) with their types, constraints, and purposes. The description does not add any meaningful parameter semantics beyond what the schema provides; it only references the 'query' parameter indirectly in usage tips. Thus, it meets the baseline score of 3, as the schema adequately covers parameter details without needing extra explanation in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search through documentation chunks using semantic search.' It specifies the verb ('search'), resource ('documentation chunks'), and method ('semantic search'), making the function unambiguous. However, without sibling tools for comparison, it cannot demonstrate differentiation, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance: it advises making queries 'specific to get the best results' and warns that forgetting to add 'api' to the query will return 'ui results etc.' This offers some context on how to use the tool effectively. However, it lacks explicit when-to-use scenarios, prerequisites, or comparisons to alternatives, as no sibling tools exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.