Server Quality Checklist

Tool Scores

• lexicon_searchA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes full responsibility for behavioral disclosure and does so thoroughly. It explains the meaning of the score field (ordinal, query-relative), the confidence field (absolute, comparable, with actionable thresholds 0.80 and 0.60-0.80), and the matched_by field, including what signals the strongest match. This goes well beyond a basic statement of behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every section earns its place: purpose, when to use, and a compact but essential guide to interpreting results. The confidence threshold guidance is front-loaded after the purpose and prevents wasted effort on weak hits. No filler or tautology is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is effectively complete for a search tool. It covers what is searched, when to search, how to interpret the returned fields, and how to handle the 'corpus not covered' case. The schema handles parameter details and an output schema exists, so the description must focus on decision-making semantics, which it does.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all six parameters adequately. The description reinforces the query technique of quoting exact identifiers, paths, or error strings, but adds no new parameter-level semantics beyond what the schema provides. This matches the baseline of 3 for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Search the Lexicon') and the specific resource scope ('curated project notes, in-place repo documentation, and archived session transcripts'). It is clear that this tool searches rather than reads, but it does not explicitly mention or differentiate the sibling tool lexicon_read, so it misses the full 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit when-to-use directive: 'Use before re-solving a nontrivial problem to find prior work, past decisions, and approaches that already failed.' It provides clear context for when the tool is appropriate, but does not state exclusions or contrast with lexicon_read, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lexicon_readA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency burden. It discloses the core behavior: reading a whole document or a centred chunk with surrounding context, and it clarifies that path must come from lexicon_search. It does not explicitly state read-only/no side effects, but the verb 'Read' strongly implies it, and the chunk/context semantics are well described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The action and scope appear first, and the follow-up sentence provides the exact usage pattern with the sibling tool. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a three-parameter tool with an output schema and a single sibling, the description is complete. It explains what the tool does, when to use it, how to connect it to lexicon_search, and how the chunking parameters behave. No critical information is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters and already explains path provenance, chunk_ord as the centre, the whole-document option, and context_chunks as each-side context. The description adds orchestration context (use after search) but does not significantly add parameter-level meaning beyond what the schema provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read') and resource ('an indexed document, or one chunk of it with surrounding context'), and clearly distinguishes it from the sibling tool by describing the read operation that follows search. It is immediately obvious that this tool is for expanding an existing search result, not for searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use after lexicon_search to expand a promising result.' It also tells the agent exactly how to invoke it, by passing the path and chunk_ord from that result, which effectively routes the agent between lexicon_read and lexicon_search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.