Server Quality Checklist

Tool Scores

• list_documentsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It implies a read-only listing operation ('List') but does not explicitly confirm safety or side effects. It also does not mention pagination, sorting, or limit semantics, although the schema's limit parameter is self-explanatory. The description adds minimal behavioral context beyond the obvious reading of 'list', but for a simple retrieval tool this is borderline adequate—hence a 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that leads with the primary action and resource, followed by a purpose clause. It contains no filler, fluff, or redundant information. Every phrase serves a function: the action, the object, and the reason. It is appropriately brief for a tool of this simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one optional parameter, output schema present), and the description covers its purpose and when to use it. The only minor omission is clarification of what 'indexed' refers to (e.g., all documents, memory entries), which could confuse an agent. However, given the output schema likely defines the structure and the tool name is clear, the description is largely complete for practical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single parameter 'limit' with default/min/max but no description, and schema description coverage is 0%. The tool description does not mention the limit parameter at all, leaving its semantics entirely to inference from the name and constraints. Since the description fails to compensate for the lack of schema documentation, the parameter meaning is not clearly communicable, particularly for agents unaware of common 'limit' conventions. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('List') and a resource ('what is indexed'), indicating it retrieves the set of indexed items. It hints at its purpose relative to siblings ('so a search can be narrowed' contrasts with search_memory, and 'gap in the memory' suggests memory_status), but it does not explicitly name or differentiate those siblings. It is more specific than a tautology and gives a meaningful overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context for when to use the tool: before a search to narrow scope, or to recognize gaps in memory. However, it does not explicitly state when NOT to use it or name alternative tools (e.g., 'use search_memory for retrieving specific content'). The guidance is implied rather than explicit, so it earns a mid-range score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_statusA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral burden. It discloses that this is a read-only status report and indicates liveness, but it does not describe error conditions, what 'bound to' means in concrete terms, or the shape of readiness reporting beyond the output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every phrase adds useful information about the tool's scope and current usefulness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no parameters and an output schema exists, so the description does not need to explain return values. It sufficiently covers the core purpose and operational state, though it could mention typical usage context such as 'call before search_memory'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are trivially satisfied. The description correctly focuses on behavior rather than inputs, and the baseline for zero-parameter tools applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Report') and names the exact resources: the bound knowledge base and service readiness. This clearly distinguishes it from sibling tools that list, search, or read documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys a clear diagnostic purpose and implies use as a pre-flight check before searches ('whether the service can answer searches right now'). It does not explicitly name alternatives or exclusions, but the context is strong enough for an agent to infer when to call it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_documentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It clearly communicates the read-only nature and discloses important edge-case behavior: offsets past either end are clamped rather than refused, and total_codepoints distinguishes clamped from exhausted ranges. This is strong for a read tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences, each earning its place: purpose, how to call it, and a key edge-case behavior. No filler or redundant restatement of the name or schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with an output schema available, the description covers what, when, how, and the main edge-case. The connection to search results makes its role among siblings clear. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description must compensate for start and end. It explains that start/end correspond to search-result offsets and recommends widening them. It adds meaning beyond the bare integer schema, but does not fully specify exact inclusivity or default-range behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('read a document's text around a character range') and a clear use case ('to see what a search result was cut off from'). It naturally differentiates itself from sibling tools like search_memory by being the follow-up expansion action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides concrete guidance: pass document_id and source.start_offset/source.end_offset, widened by a few hundred characters. It implies the intended workflow after a search result, though it does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_memoryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It reveals that ranking is vector similarity, explains why first results can be misleading, and names the exact failure mode for rerank (RERANK_NOT_CONFIGURED). This goes well beyond a generic search description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and each subsequent sentence earns its place: query phrasing, retrieval strategy, and the rerank pitfall. It is dense but not bloated, and the structure guides the agent from what to do to what to avoid.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, an output schema is present, and the description covers the key decision points: query formulation, retrieval volume, and rerank configuration. Nothing essential for calling this tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 50% schema coverage, the description compensates well for query and rerank semantics, and it implicitly motivates top_k through 'retrieving more than you need.' source_type is not addressed in the description, but the schema already provides an explanatory description and example for it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a clear resource ('indexed memory'), and the outcome ('return the passages that match'). This clearly distinguishes search_memory from siblings like read_document and list_documents even without reading the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete usage guidance: prefer a specific question, retrieve more than needed, and leave rerank off unless a profile is configured. It does not explicitly compare against sibling tools or state when not to use search_memory, but the guidance is strong and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.