Server Quality Checklist

Tool Scores

• rag_searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral burden. It clearly discloses the read-only nature ('never modifies the index') and the output structure (chunks, source paths, indices, scores). It also notes the empty‑result behavior and suggests a follow‑up action. It omits potential error conditions or performance nuances, but the disclosure is substantial for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized—four sentences, each contributing distinct information: mechanism, output, read‑only guarantee, and usage routing. It is front-loaded with the core purpose and avoids redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description supplies essential return information (file paths, chunk indices, scores) and explains the retrieval approach. It also addresses the empty‑index scenario and distinguishes from sibling tools. Minor gaps (e.g., exact result JSON structure, authentication) are not critical for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters. The description reiterates the role of top_k ('up to top_k ranked chunks') but does not add syntax, formats, or constraints beyond the schema's own descriptions ('default 5, max 25'). The hybrid retrieval detail is tool-level, not parameter-specific. Minimal added value per baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Search the local knowledge base'), the retrieval method (hybrid), and the output (ranked chunks with file paths, indices, scores). It explicitly contrasts with sibling tools by noting 'Prefer this over rag_status when answering a user's question about their documents', making the tool's role unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: when to use (answering a user's question about documents), when not to (prefer rag_status for other purposes), and a conditional fallback ('If results are empty, call rag_index first to populate the knowledge base'). This fully equips the agent to choose and sequence the tool correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rag_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool is read-only, instant, and does not load the embedding model. This is good behavior disclosure for a report tool, though it doesn't mention potential limitations like staleness or exact response format, which is minor given the simplicity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The first sentence lists the precise report contents, and the second gives usage and behavioral context. Information is front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description fully covers what it does, what it returns (stats and breakdown), and when to use it. Nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description doesn't need to explain parameters; it appropriately stays silent on them. No additional meaning is required beyond what the schema (empty) provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports knowledge base statistics (document count, chunk count, total indexed bytes, per-file-type breakdown). It uses a specific verb and resource, and the content distinguishes it from sibling tools like rag_search, rag_clear, and rag_index.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use it 'to check whether anything is indexed before running a search,' providing a clear context of use. It doesn't explicitly list when not to use it or alternatives, but the intended use case is clear and the siblings are evident from the tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rag_clearA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since there are no annotations, the description carries the full burden of behavioral disclosure. It explicitly states the operation is destructive and permanent, describes what is deleted (documents, chunks, embeddings), what is not deleted (source files on disk), and the necessity of re-indexing. It also includes a confirmation requirement. This is exemplary transparency for a destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the critical warning 'Destructive'. Each sentence adds essential information: scope, permanence, what is preserved, and the confirmation requirement. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema destructive tool, the description covers all necessary aspects: what it does, consequences, side effects, and a required user interaction. An agent has everything needed to call it correctly and safely. The absence of an output schema does not matter since no output is described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain beyond the schema. The baseline for 0-parameter tools is 4, and the description does add relevant context about the scale of the deletion, which is useful. No additional parameter details are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'delete' and the resource (all indexed documents, chunks, and embeddings from the knowledge base). It distinguishes this tool from siblings like rag_search, rag_status, and rag_index by emphasizing it is a destructive clear-all operation. The scope is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context (destructive, permanent) and an explicit instruction to confirm with the user. It also notes that re-indexing is required afterwards, which implies it should only be used when a full reset is intended. However, it does not explicitly name alternatives or say when not to use it, but the destructive nature makes the usage context obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rag_indexA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly covers the tool's behavior: recursive incremental walk, content-hash change detection, deletion pruning, idempotency, re-embedding only changed content, and automatic skipping of dot-directories. No contradictions with any structured metadata exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (four sentences), front-loaded with the primary purpose, and each sentence contributes meaningful information. No redundancy or filler; it efficiently packs behavioral details and sibling references without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately explains the tool's actions and effects for an indexing operation, given that no output schema exists. It mentions the workflow step ('After indexing completes, use rag_search') but does not specify the response format or return value. This is a minor gap; for a tool like this, the key context is what gets destroyed/created and when to use it, which is well covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters. The description adds value by clarifying that 'path' accepts an absolute directory or a single file, and mentions supported types (md/txt/code) which relates to the 'extensions' parameter's defaults. This goes beyond the schema's basic explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Index') and resource ('local documents into the knowledge base'), and explicitly distinguishes itself from siblings by noting that rag_search is for querying and rag_clear is for removal. This makes the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage context: 'before searching', and directs users to use rag_search after indexing and rag_clear for removal. This effectively routes the agent to the correct workflow and alternatives, leaving no ambiguity about when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.