Server Quality Checklist

Tool Scores

• browse_manualA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates that this is a cheap, non-destructive orientation tool ('Cheapest way', 'list sections'). It discloses what it filters by and discloses its return granularity (sections/headings, not full passages). The depth is good given zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core purpose, and every sentence earns its place. The strategy framing (cheapest, recovery move) and sibling comparison are woven into two tight lines, followed by a minimal Args block. Zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 3-parameter browsing tool with an output schema present, the description is complete. It explains what it does, why to use it, when to prefer it over the key sibling, and clarifies all parameter semantics. The presence of an output schema relieves it from describing return values. Nothing material is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters. It does: 'query' gets keyword-filter semantics (section headings and page titles), 'family' gets an operator-family filter with the enumerated options, and 'limit' gets maximum-section semantics. This fully compensates for the 0% coverage, though exact value formats aren't elaborated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb+resource ('list page sections') with a defined scope ('to orient before retrieving full passages'). It distinguishes itself from siblings by positioning it as a discovery/browsing tool versus search_manual. The 'L0' prefix and vocabulary-discovery framing make its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly compares to search_manual ('Prefer this over search_manual when you need the shape of a topic rather than a specific answer'), giving a clear when-to-use directive. It also states the recovery use case ('best recovery move when a search misses'). It doesn't discuss when NOT to use it relative to read_page, but the when-to-use guidance relative to the key sibling is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the L2 classification (suggesting a tiered access model), the exact-title matching behavior, and the section-scoping capability. It doesn't detail error behavior for nonexistent titles, but for a simple read with an output schema and no safety-critical behavior, this is largely sufficient. A 'read' operation's mutability is implicit in the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence earns its place: a one-line purpose, context positioning, title-format clarification with examples, and a compact args block. Zero waste, well front-loaded. The markdown structure mirrors the args list cleanly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with an output schema, the description is nearly complete. It adds positioning among siblings, exact-title requirements, and section behavior. Minor gap: no statement about behavior when section doesn't exist or when title is invalid, and the L2 reference is unexplained, but overall the tool is adequately specified for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate and does. It explains that title must be an exact wiki title with concrete format examples, and explains section is optional and returns just that section. Both parameters get semantic context beyond bare names/null defaults. It doesn't specify section heading exactness rules, but the core semantics are well covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads the full markdown of one wiki page or one of its sections, using a specific verb ('read') with a clear resource ('wiki page'). It gives concrete example titles ('Noise TOP', 'CHOP Execute DAT') and explicitly distinguishes from sibling tools by referencing search_manual and browse_manual as preceding steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: 'Use after search_manual or browse_manual to get surrounding context.' It also establishes the exact-title matching requirement and gives examples of valid titles, which prevents common agent errors like fuzzy matching. No exclusions needed since the tool is a straightforward read.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_manualA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers: it discloses the multi-stage retrieval process, the automatic HyDE second pass when a miss is suspected, the citation output ('wiki-URL citations'), and that it handles both exact names and paraphrases. This is rich behavioral context well beyond what a schema would convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a header line, retrieval-technique summary, and clear per-parameter annotations. Every sentence earns its place, technical detail is front-loaded, and the format is scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (hybrid retrieval, fusion, rescoring, conditional HyDE), and the description covers this well. There is an output schema, so return-value explanation isn't required. The only slight gap is whether the 'L1' tag and end-user intent guidance could be clearer, but for a single-free-text-query retrieval tool this is quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters. It does well: query ('What you need to know'), family (with concrete enum-like values and usage guidance), and limit ('Maximum passages to return'). Only minor gap is no detail on limit's default or range, but overall the description adds strong meaning absent from the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is specific: 'find passages answering a question, with wiki-URL citations' plus a 'L1' label. It clearly identifies the resource (manual/search passages) and the action (retrieve answers). Though siblings include browse_manual and read_page, the search/retrieve semantics are distinguishable from browsing or reading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains the retrieval pipeline (BM25, dense, RRF fusion, cross-encoder rescoring), notes the automated HyDE fallback on probable misses, and tells when to use the family parameter ('when the question is about one family's operator; same-named operators exist across families'). This gives a clear context and exclusions for parameter usage, though it doesn't explicitly name when NOT to use this vs siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.