Server Quality Checklist

Tool Scores

• rerank_resultsA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that unknown doc_ids are silently skipped and that scores are not comparable to search_documents scores, which is helpful. However, it doesn't detail side effects (none expected) or performance implications beyond general compute cost, which is mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but thorough, with front-loaded purpose, clear usage guidance, parameter details, and return format. Every sentence adds value without redundancy, despite being longer than ideal, it remains structured and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a tool with two simple params, no annotations, and a clear output schema. It explains the return format and score semantics, covers edge cases (unknown ids), and contextualizes the tool's role in the workflow, making it self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides meaningful semantics for both parameters: query is the reference for re-ranking, doc_ids is the candidate list and indicates unknown ids are skipped. This adds value beyond the minimal schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool re-ranks a provided candidate list relative to a query using a cross-encoder, distinguishing it from direct similarity search. It explicitly states it does not find new documents, differentiating it from sibling tools like search_documents and get_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use (when a candidate list already exists) and when-not-to-use (for corpus-wide search, use search_documents). Mentions alternatives and gives a specific scenario (before quoting top-N to the user), making usage guidance clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden for behavioral disclosure. It explicitly mentions the graceful error handling: if doc_id is not found, it returns a list of available doc_ids instead of an error, which is key behavioral context. However, it does not mention potential performance implications (e.g., fetching a large document) or any authentication requirements, so it is not perfect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise, with a clear first sentence stating the main purpose, followed by usage guidance and parameter/returns details. It is structured in sections (Args, Returns) for readability. However, the 'Args:' section is somewhat redundant with the schema, and the text could be slightly more streamlined without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no annotations) and the presence of an output schema, the description is complete. It covers purpose, usage context, parameter semantics, and return behavior. The graceful error handling is a nice touch that prepares the agent for a non-standard response, making it comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines doc_id as a string but provides no documentation. The description adds significant meaning by explaining that doc_id matches the field returned by search_documents() and corresponds to the filename in the corpus (e.g., 'hybrid-search.md'), which is not evident from the schema. With 0% schema description coverage, this compensation is crucial and well-executed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving the full text of a document from a corpus by its doc_id. It uses a specific verb ('Возвращает') and resource ('документ из корпуса'), and it distinguishes itself from siblings by focusing on fetching full text, while search_documents is for searching snippets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: after search_documents() when the snippet is insufficient, and provides concrete examples (quoting exact phrasing or reading sections not in the snippet). It also names the sibling tool search_documents as the prior step, making the workflow clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provisioned, the description carries the full explanation burden. It does well by disclosing the dense+BM25 RRF behavior, a relevance-sorted return structure, and downstream integration via doc_id to get_document or rerank_results. It maybe does not explicitly state side-effect safety or permissions, but such considerations are minimal for a search operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well structured: a front-loaded explanation of the retrieval mechanism, followed by concrete usage guidance and a neat Args/Returns block. Every sentence completes the main explanation of what the tool does, when to use it, how it behaves, and what the caller receives.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, sibling tools, and search-oriented complexity, the description provides a practically complete flow: search for candidates, receive relevant snippets and scores, then optionally pass doc_id to get_document for full content or rerank_results for reranking. This is sufficient contextual guidance for agentic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While the schema description coverage is 0%, the description fully explains both parameters: query is a natural-language request or keywords, and top_k is the number of documents to return with a default of 5 and maximum of 20. This adds strong semantic value beyond the bare input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: hybrid search over an indexed corpus of markdown documents about RAG, retrieval, and LLM infrastructure. It distinguishes this tool from siblings like get_document and rerank_results by making search and retrieval its primary role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit direction: use this tool to find relevant documents or facts before answering instead of relying on internal knowledge. It also explains suitability for semantic queries and exact terms; however, it does not explicitly describe when not to use this tool versus its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.