Server Quality Checklist

Tool Scores

• list_productsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It adds valuable traits beyond a plain list operation: the call is 'cheap and cacheable' and results only change when the index is rebuilt. For a read-only listing tool, this is useful context, though it could go further on pagination or output size expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the first explains output scope, the second gives a usage trigger with a warning, and the third provides cost and freshness context. It is front-loaded with the most important information and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema and fully documented optional parameters, the description covers the remaining context: what data is returned, when to call it, and why it is safe/cheap to call. Nothing an agent needs to decide whether to invoke this tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema describes both parameters completely, so the baseline is 3. The description does not add parameter-level details beyond saying 'every product', which is slightly at odds with the optional filters, but the schema itself is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource ('List every Neutrinos product'), and enumerates exactly what the result includes: versions, current version, topic count, and last-updated time. This is distinct from the sibling tools like 'search_docs' and 'compare_versions', so an agent can select it confidently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to call this tool: before 'search_docs' when unsure what to pass for 'product' or 'version'. It even explains the failure mode ('guessing filter values is the most common cause of an empty search'), which gives a clear, actionable decision rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_relatedA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains what the tool returns, the variety of relation types, page-level scoping, and even interprets the semantic meaning of `prev`. It doesn't discuss errors or auth, but with an output schema present and no risky side effects indicated, the key behavior is sufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and resource in the first sentence. The second sentence gives concrete use cases, and the third provides a terse but valuable hint about `prev`. No sentence is wasted, and the length is appropriate for the number of relation types covered.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of a detailed input schema, an output schema, and a moderately complex set of relation kinds, the description is complete enough for an agent to select and call the tool correctly. It explains the tool's purpose, when to use it, and adds practical relation semantics without needing to restate structured schema details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers all three parameters fully at 100% coverage, so the baseline is 3. The description adds value beyond the schema by explaining the practical meaning of `prev` in the context of prerequisites and by framing the overall result as a 'typed neighbourhood', giving semantic cohesion to the relation enums.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Return the typed neighbourhood of a documentation page' and then enumerates the exact relation kinds returned. This clearly distinguishes the tool from siblings like search_docs, fetch_document, compare_versions, and list_products, which concern searching, fetching content, comparing versions, and listing products respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete use cases: when a page assumes a prerequisite you have not read, or when you need to confirm which product versions document a behaviour. It also provides a practical hint that `prev` is the best signal for prerequisites. It does not explicitly mention when not to use the tool or name alternatives, so it stops just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it discloses scope, version-sensitive behavior (only current versions unless product/version passed), and the meaning of sufficient_evidence. It could add a bit more about limits (e.g., rate, token cost, or that it is a read-only search), but the disclosed behaviors are genuinely important and not obvious from the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, tightly packed, front-loading the core purpose before caveats. Every sentence adds actionable value: what it searches, when to use it, how to handle versions, and how to interpret the evidence flag. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for an agent facing a documentation-search task: it states scope, expected behavior, version handling, evidence interpretation, and a sibling fallback. The output schema exists, so return values need no elaboration. This is a well-rounded definition that would let an agent call the tool correctly in nearly all scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all six parameters, including examples and defaults. The description adds context for product and version (version sensitivity) and references list_products for values, but it does not materially enrich the meaning of query, top_k, response_format, or include_superseded beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a concrete resource (Neutrinos product documentation with corpus counts), and a clear output ('ranked, citable passages'). It positions itself as 'the entry point for any question about Neutrinos products,' distinguishing it from siblings like fetch_document and list_products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly frames when to use the tool ('any question about Neutrinos products'), gives conditional guidance (pass product/version when version is known), and points to a sibling (call list_products first if unsure). It also tells the agent what to do with the result (check sufficient_evidence), leaving no ambiguity about when to apply this search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_documentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It reveals that responses are hard-capped at max_tokens and that page length can make whole-page retrieval impractical. It could mention continuation or failure behavior, but the output schema and max_tokens parameter partially cover that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all substantive: purpose, usage trigger, and a concrete recommendation. No filler, no repetition, and the core information is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a full output schema and 100% parameter documentation, the description supplies the essential invocation context: when to call, how to choose between page and section, and the response cap. It does not discuss alternatives like list_related or compare_versions, but those are less relevant to correctly invoking this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all four parameters with 100% coverage, so the baseline is 3. The description adds meaning beyond the schema by naming the source of ref, explaining why section is preferable, and highlighting the practical consequence of max_tokens.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States the exact operation: retrieve the full text of a documentation page or one section by ref. It explicitly ties ref to search_docs results, distinguishing this tool from the search_docs sibling and other documentation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives an explicit trigger: use when a search passage is clearly relevant but cut off mid-explanation or mid-code-block. It also advises preferring the section parameter because whole pages can be long and output is hard-capped at max_tokens.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_versionsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It discloses an important non-obvious behavior: the tool follows product renames and spans them where name-based search would not. It does not explicitly say whether the operation is read-only or what happens when no versions match, but the comparison semantics are clear from the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three focused sentences: what the tool does, when to use it, and a relevant caveat. The most useful routing information is front-loaded, and no space is wasted on redundant restatements of the schema or annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the parameter schema fully documents all four parameters and an output schema exists, the description provides the missing contextual pieces: use-case triggers, relationship to `search_docs`, and rename-following behavior. This is sufficient for an agent to decide when to invoke the tool and what parameters mean in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds extra semantic value beyond the schema by explaining that renamed products (e.g., App Builder becoming Studio) are followed, which directly informs how `product` and `versions` parameters should be interpreted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Show') and resource ('how one documentation topic differs between product versions, section by section'), making the core function immediately clear. It also references sibling `search_docs` and its `version_ambiguous` output, which differentiates this tool from related search/fetch tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit when-to-use guidance is provided: use for 'this worked in version N but not N+1' questions and before answering when `search_docs` set `version_ambiguous`. This tells the agent exactly the triggering conditions and distinguishes the tool from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.