Server Quality Checklist

Tool Scores

• search_documentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses output shape (ranked snippets with locators), a key limitation (never full text), and a recommended next action. It doesn't mention pagination or error conditions, but for a search tool the provided behavioral details are substantial and accurate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first defines the search mechanism, second specifies the output and locators, third gives a crucial usage boundary and points to the sibling. Perfectly front-loaded with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The overall context is adequate for a simple search invocation: it explains output, the usual starting-point role, and the follow-up workflow. However, with 5 parameters including a nested filter object and no annotations, the description leaves mode/filter/scoping semantics unexplained, so it's not fully complete despite the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20%, so the description must compensate. It only vaguely references the hybrid mode and the fact that search spans 'ingested documents'; it does not explain k, mode choices, the filter object, or how document_id scoping works. The description adds almost no value for the parameter set beyond what the bare schema shows.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs 'Hybrid BM25 + semantic search across ingested documents' and returns 'ranked snippets with precise locators' — a specific verb+resource. It distinguishes itself from siblings by emphasizing search and pointing to get_chunk_context as the follow-up for full text, so an agent knows this is the retrieval entry point.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'This is the usual starting point' and provides an exclusion: 'It never returns full text — follow a hit with get_chunk_context using its chunk_id to read.' This tells the agent when to use this tool and when to switch to a named alternative, which is exactly what this dimension asks for.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_chunk_contextA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the hard cap of ~24k characters, the neighbor window of up to 5 on each side, and the progressive reading strategy. This is valuable behavioral context beyond the schema, though it does not cover error cases or permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences with no redundant words. The most important function is front-loaded, followed by quick addressing instructions and a practical cap warning. Perfectly sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (so return structure need not be detailed) and the tool's moderate complexity, the description covers all essentials: what it returns, how to address chunks, neighbor bounds, the character cap, and progressive reading advice. No critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 40%, but the description fills gaps by explaining the two addressing modes (chunk_id OR document_id+seq) and the neighbor limit, which maps to after/before parameters. It adds meaning beyond the bare schema, especially for uncovered params like seq and after/before.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'Full text of one chunk plus up to 5 neighbours', which is a specific verb+resource+scope. It explicitly distinguishes itself from siblings by noting 'This is the only tool that returns body text', leaving no ambiguity about its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use it (when body text is needed) and how to address chunks via chunk_id or document_id+seq, with sources for those IDs. It also advises 'walk seq windows to read progressively' for large reads, though it does not explicitly state when not to use it or name alternatives beyond implying it's the only body-text tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ingest_documentA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility and excels. It discloses asynchronous indexing, immediate return of document_id, background processing, OCR for scanned PDFs, content-based format detection, and idempotent re-ingestion—far exceeding basic safety/behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph where every sentence adds substantive value. It front-loads the core purpose, then logically covers formats, OCR, format detection, async behavior, and re-ingestion—no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (async indexing, OCR, format nuances) and absence of annotations, the description is remarkably complete. It even explains how to verify completion via get_document_outline, making it self-sufficient for an agent. The output schema covers return structure, so no further detail is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for both parameters ('Path relative to the library root' and 'Overrides the title detected from the file'). The tool description adds no additional parameter-level insight, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Index a file from the library into the retrieval index,' clearly stating the verb and resource. It distinguishes this from siblings like search_document and delete_document by focusing exclusively on ingestion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for use, including supported formats, refusal behavior, OCR handling, and re-ingest semantics. It does not explicitly contrast with alternatives, but the sibling tools are obviously different operations, so the guidance is adequate without formal exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_documentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully discloses the destructive nature of the operation, states that the source file is untouched, and warns that deletion is permanent and chunk_ids will stop resolving. It also explains that re-ingesting rebuilds the document, giving a complete behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured, and every sentence adds meaningful information: the primary effect, the scope, the typical use case, the exception for edits, and the permanence/consequence. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple one-parameter schema and its rich explanatory description, the description is fully complete for an agent to decide when and how to use it. The presence of an output schema also means return-value details are not required in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the single parameter (document_id) with a description referencing get_document_outline's library listing. The description does not add further param-specific meaning, but the schema already provides the necessary context, so a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Remove a document from the index') and lists exactly what is removed (chunks, full-text entries, vectors). It also distinguishes this from sibling tools like ingest_document by explicitly noting that re-ingesting replaces versions, so delete is only for mistaken ingestion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance ('Use it to drop something ingested by mistake') and an exclusions ('an edited file does NOT need this, because re-ingesting one already replaces the version it supersedes'). This clearly directs the agent away from unnecessary deletion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_outlineA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses low context cost, no body text return, and dynamic ingest progress reporting (status 'processing' with rising chunk_count). This is strong transparency beyond any structured metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each adding unique value: the return type, the usage context, and the progress-reporting behavior. Front-loaded with the core purpose and no filler words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers output nature (headings, locators, chunk seq spans), a key limitation (no body text), and dynamic status reporting. Combined with output schema and sibling context, it gives an agent sufficient understanding to invoke correctly despite the max_depth omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema documents document_id with 'Omit to list every ingested document with its status' but max_depth lacks description. Description reinforces document_id behavior but never explains max_depth or its effect on outline depth. With 50% schema coverage, this only partially compensates for the missing parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as returning a hierarchical heading tree with locators and chunk seq spans. It distinctively separates this from sibling get_chunk_context by stating it never returns body text and is for orientation before targeted content reads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: use to orient before get_chunk_context reads, call with no document_id to list the library, and avoid expecting body text. This tells when and how to invoke the tool, and implies when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.