Server Quality Checklist

Tool Scores

• add_documentsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits beyond the basic action. It does not mention whether the operation is idempotent, requires authentication, overwrites existing documents, or how it handles duplicates—critical context for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler or redundant information. It front-loads the primary action and then clarifies the optional parameter in a direct, efficient manner.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with only two parameters and an output schema exists, so the description does not need to explain return values. However, the lack of behavioral transparency (e.g., side effects, permissions) leaves gaps for a write operation, making the description minimally complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds value by explaining that 'text' is the document content and 'source' is a label for provenance. This goes beyond the schema's type and default information, providing meaningful semantic context for both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: adding a text document to the knowledge base. It uses a specific verb ('Add') and resource ('knowledge base'), and the purpose is distinct from sibling tools that query, search, or approve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool is for adding documents, implying it should be used when you want to insert new content rather than retrieve or edit existing content. It does not explicitly mention alternatives or exclusions, but the purpose is self-evident enough to guide usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_or_edit_answerA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explains the decision logic (approved=True/False) and that user_feedback is needed for edits, which is useful. However, it does not disclose side effects, whether approval modifies the knowledge base, or any permissions/irreversibility. The tool's outcome beyond the parameters is left implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the core purpose ('Human-in-the-loop') and immediately provides the usage pattern. Every phrase contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 params, 2 required) and the existence of an output schema, the description is adequate but not rich. It covers the branching logic and references the source (query_knowledge_base), but omits preconditions, post-conditions, or what happens next in the pipeline. This leaves room for agent confusion about workflow integration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'approved' (accept or request changes) and 'user_feedback' (changes to request), but 'proposal_answer' is only indirectly referenced as 'the proposed answer'. While this adds meaning beyond the empty schema, one parameter remains underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to approve or request edits for an answer proposed by query_knowledge_base. It uses a specific verb ('approve', 'request edits') and identifies the resource ('the proposed answer'), distinguishing it from sibling tools like query_knowledge_base, add_documents, and search_knowledge_base.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: after obtaining a proposal from query_knowledge_base, this tool decides on acceptance or revision. It gives clear context ('Human-in-the-loop') and ties the tool to a parent process, but does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_knowledge_baseA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the key behavioral trait that this is a retriever-only tool returning chunks, which is significant. However, it does not detail output format or any limitations, but the output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded and contains no unnecessary words. It effectively communicates the core behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with an output schema, the description is fairly complete. It clearly states the retriever-only nature and what is returned (chunks), but could more explicitly address when to use it over siblings. Still, the phrase 'only (retriever)' provides a hint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain what 'query' or 'top_k' mean. While 'query' is self-evident from the tool name, 'top_k' is ambiguous. The description provides no additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the knowledge base and returns chunks without generating an answer. This distinguishes it from sibling tools like query_knowledge_base, which presumably generates answers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool ('only retriever', 'without generating an answer') but does not explicitly name alternatives or provide exclusion criteria. It gives clear context, though a direct pointer to query_knowledge_base would be stronger.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_knowledge_baseA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes beyond a simple query by revealing the multi-agent orchestration, the fact that the answer is only a proposal, and the human-in-the-loop requirement. This gives the agent a clear understanding of the tool's non-final and collaborative nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, each serving a distinct purpose: stating the action, explaining the internal pipeline, and providing the next-step workflow. It is front-loaded with the primary purpose and contains no redundant or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is largely complete for a query tool: it covers the action, the pipeline, and the follow-up workflow. The existence of an output schema means return values need not be detailed. However, the missing parameter semantics for 'top_k' and the lack of explicit contrast with search_knowledge_base leave minor gaps in overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The 'query' parameter is implicitly explained as the question to ask, but 'top_k' is not mentioned at all, leaving its role (e.g., number of retrieved documents) ambiguous. This is a significant gap given the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Ask') and resource ('knowledge assistant'), and clearly explains the multi-agent pipeline (coordinator -> retriever -> synthesizer). It distinguishes the tool from siblings by mentioning it returns a proposed answer for review, which is unlike a direct search or an approval tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear workflow context by instructing the user to call approve_or_edit_answer after reviewing, which implicitly indicates a sequential use. However, it does not explicitly differentiate when to use this tool versus search_knowledge_base, missing the 'when-not' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.