Server Quality Checklist

Tool Scores

• list_sourcesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It clearly indicates a read-only listing operation, and explicitly mentions the returned data (file and chunk counts). Despite lacking details on edge cases like empty index or error handling, the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that names the tool's purpose and data included. It avoids unnecessary words and is well-front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter listing tool with no output schema, the description covers the essential information: what is listed and what counts are included. It lacks details on ordering, pagination, or empty states, but these are not critical for such a basic tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema provides no properties. The description does not need to parameter semantics, and the baseline for 0 params is 4. The description adds no parameter information, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists folders currently indexed in the user's local document index, including file and chunk counts. This is a specific verb-resource combination that distinguishes it from the sibling search_documents tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this to see what folders are indexed, versus search_documents for searching content. However, it does not explicitly state when or why to choose this tool over alternatives, nor does it mention exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the search method (hybrid keyword + semantic), the scope (local document index), and the output shape (most relevant passages with file-and-page citations). It does not cover edge cases or performance characteristics, but it is reasonably transparent for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences. The first sentence front-loads the core purpose and signature, while the second gives a clear usage trigger. Every sentence earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple search tool with a complete input schema and no output schema, the description covers the essential scope, method, output, and use case. It could mention pagination or default limit behaviors, but these are already defined in the schema, so the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the 3 parameters is 100%, and each has descriptive meaning in the schema (e.g., 'What to search for'). The description adds no additional parameter semantics beyond what the schema already provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'user's local document index' with specific detail about hybrid keyword + semantic search. It naturally distinguishes itself from the sibling tool 'list_sources', which lists sources rather than searching content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance with 'Use this whenever a question might be answered by the user's own files,' but does not explicitly mention when not to use it or name alternatives. The contrast with list_sources is implied, but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.