Server Quality Checklist

Tool Scores

• query_knowledge_baseD
  1.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but offers none. It doesn't indicate whether this is a read-only operation, what the response format might be, whether there are rate limits or authentication requirements, or how results are returned (e.g., relevance-ranked). For a query tool with 3 parameters and no output schema, this leaves critical behavior undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While technically concise with just three Chinese characters, this is a case of severe under-specification rather than effective conciseness. The description doesn't earn its place - it provides no useful information beyond the tool name itself. Good conciseness balances brevity with information value, which this completely lacks.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (query operation with 3 parameters), absence of annotations, and lack of output schema, the description is completely inadequate. It doesn't explain what the tool returns, how results are structured, what the threshold parameter means in practice, or any behavioral characteristics. For a query tool that presumably returns ranked results, this leaves the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with all parameters documented in the schema itself ('question', 'max_results', 'threshold'). The description adds no additional meaning beyond what the schema provides about parameter purposes or usage. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no parameter information in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '查询知识库' (Query knowledge base) is a tautology that essentially restates the tool name 'query_knowledge_base' in Chinese. It provides no specific verb or resource details, doesn't explain what kind of querying occurs (e.g., semantic search, keyword matching), and doesn't distinguish this from sibling tools like 'get_document' or 'list_documents' that also retrieve information from the knowledge base.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides absolutely no guidance on when to use this tool versus alternatives. There's no mention of when this tool is appropriate (e.g., for natural language questions vs. structured queries) or when to use sibling tools like 'get_document' (for specific documents) or 'list_documents' (for browsing). The agent receives no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_knowledge_baseD
  1.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. '清空知识库' implies a destructive mutation (clearing), but it doesn't specify whether this action is reversible, requires special permissions, affects system performance, or has confirmation steps. For a potentially high-impact tool with zero annotation coverage, this lack of behavioral context is a critical gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single phrase '清空知识库', which is extremely brief. While conciseness is valued, this under-specifies the tool's function—it lacks necessary context about behavior, usage, or effects. Every sentence should earn its place, but here the minimal content fails to provide adequate information, making it inefficient rather than appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's likely complexity (a destructive clear operation) and the absence of both annotations and an output schema, the description is incomplete. It doesn't explain what 'clear' entails (e.g., deletion of all documents, resetting statistics), what happens post-operation, or any error conditions. For a high-stakes tool with no structured support, the description should do more to inform safe and correct use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description doesn't add parameter details, but since there are no parameters to explain, this is acceptable. The baseline for 0 parameters is 4, as the description doesn't need to compensate for missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '清空知识库' (Clear knowledge base) is a tautology that essentially restates the tool name 'clear_knowledge_base' in Chinese. While it indicates the action (clear) and target (knowledge base), it doesn't provide any additional specificity about what 'clear' entails or how this differs from sibling tools like 'remove_document' or 'query_knowledge_base'. The purpose is recognizable but minimally differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether the knowledge base must exist), exclusions (e.g., irreversible effects), or comparisons to sibling tools like 'remove_document' (for deleting individual items) or 'list_documents' (for viewing contents). Users are left to infer usage context entirely on their own.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('get') but doesn't disclose traits like read-only nature (implied by 'get'), authentication needs, rate limits, error handling, or return format. The description is minimal and lacks behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('获取特定文档信息') that is front-loaded and wastes no words. It directly states the tool's purpose without unnecessary elaboration, making it highly concise and well-structured for its minimal content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with one parameter) and lack of annotations and output schema, the description is incomplete. It doesn't explain what information is returned, potential errors, or how it differs from sibling tools. For a tool with no structured output or behavioral hints, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with 'document_id' clearly documented as '文档ID' (document ID). The description doesn't add any parameter semantics beyond what the schema provides, such as format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '获取特定文档信息' (Get specific document information) states the verb (get) and resource (document information), but it's vague about what 'information' includes and doesn't differentiate from sibling tools like 'list_documents' or 'query_knowledge_base'. It's a basic functional statement without specificity about scope or content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, exclusions, or compare to siblings such as 'list_documents' for multiple documents or 'query_knowledge_base' for search-based retrieval. Usage is implied only by the name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it 'gets' statistics, implying a read-only operation, but doesn't specify what statistics are returned (e.g., counts, sizes, types), whether it requires permissions, or any rate limits. This leaves significant gaps for a tool that likely provides critical system information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese ('获取知识库统计信息'), which is appropriately sized for a simple tool. It's front-loaded with the core action, though it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's likely complexity (providing statistics for a knowledge base with multiple sibling tools), no annotations, and no output schema, the description is incomplete. It doesn't explain what statistics are returned, their format, or how this tool fits into the broader context of managing the knowledge base, leaving the agent with insufficient guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add param info, but that's appropriate here. Baseline is 4 for 0 params, as it avoids unnecessary detail and matches the schema's simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '获取知识库统计信息' (Get knowledge base statistics) states a clear verb ('get') and resource ('knowledge base statistics'), but it's somewhat vague about what specific statistics are retrieved. It doesn't differentiate from sibling tools like 'list_documents' or 'query_knowledge_base' that might also provide statistical information, though it's more specific than just 'process'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, such as whether it's for summary metrics versus detailed listings, or how it compares to siblings like 'list_documents' for counting documents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_directoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action but doesn't describe what happens during processing (e.g., file format support, error handling, overwrite behavior, or whether this is a batch operation). For a tool that modifies the knowledge base, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese that directly states the tool's function. There's zero wasted language, and it's appropriately front-loaded with the core action. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that modifies the knowledge base (implied by 'add'), with no annotations and no output schema, the description is incomplete. It doesn't address what happens to existing documents, what formats are supported, whether this is recursive, or what the return value might be. Given the complexity of batch document addition, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'directory_path' already documented in the schema. The description doesn't add any additional meaning about the parameter beyond what the schema provides (it doesn't specify format requirements, path examples, or constraints). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('添加' - add) and resource ('目录中的所有文档到知识库' - all documents in a directory to the knowledge base). It distinguishes from sibling tools like add_document (which likely adds individual documents) by specifying bulk addition from a directory. However, it doesn't explicitly mention the scope difference compared to add_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like add_document. There's no mention of prerequisites, constraints, or comparative scenarios. The agent must infer usage from the tool name and description alone without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_documentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It implies a write operation ('添加' meaning 'add') but doesn't disclose permissions needed, whether it overwrites existing documents, error handling, or response format. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese that directly states the tool's function without unnecessary words. It's appropriately sized for a simple tool, though it could be more structured with context or examples.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., idempotency, side effects), usage context, or what happens post-addition, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents the 'file_path' parameter adequately. The description adds no additional meaning beyond what the schema provides, such as file format constraints or path examples, but this is acceptable given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('添加' meaning 'add') and resource ('文档到知识库' meaning 'document to knowledge base'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'add_directory' or 'remove_document', which would require more specificity about scope or constraints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., file must exist), exclusions (e.g., not for directories), or comparisons to siblings like 'add_directory' for bulk operations or 'remove_document' for deletions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_documentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('移除/remove') which implies a destructive mutation, but doesn't clarify if removal is permanent, reversible, requires specific permissions, or has side effects (e.g., affecting related data). The description lacks critical context for a mutation tool, leaving behavioral traits unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Chinese ('从知识库中移除文档') that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it highly concise and well-structured for its simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a destructive mutation tool with no annotations and no output schema, the description is incomplete. It doesn't address key contextual aspects like error conditions, return values, or safety warnings (e.g., irreversible deletion). For a tool that removes documents, more information is needed to ensure proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'document_id' documented in the schema as '文档ID'. The description adds no additional meaning about the parameter beyond what the schema provides, such as format examples or sourcing instructions. Baseline 3 is appropriate since the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '从知识库中移除文档' clearly states the action (移除/remove) and target resource (文档/document from knowledge base), which is specific and unambiguous. It doesn't explicitly differentiate from sibling tools like 'clear_knowledge_base' or 'add_document', but the verb+resource combination makes the purpose evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid document_id), exclusions (e.g., not for bulk removal), or comparisons to siblings like 'clear_knowledge_base' for mass deletion. Usage is implied by the action but not explicitly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists all documents, implying a read-only operation, but doesn't mention any behavioral traits such as permissions required, pagination, rate limits, or what the output format looks like (e.g., list of document names or full metadata). This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence in Chinese: '列出知识库中的所有文档' (List all documents in the knowledge base). It's front-loaded with the core action and resource, with zero wasted words or redundant information. This is optimally concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is minimal but adequate for basic understanding. However, it lacks context about the output (e.g., what data is returned, format, or limitations like pagination), which is important since there's no output schema. For a list operation, more detail on behavioral aspects would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100% (as there are no parameters to describe). The description doesn't need to add parameter semantics, so it meets the baseline expectation for a parameterless tool. No additional value is required beyond stating the purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb '列出' (list) and the resource '知识库中的所有文档' (all documents in the knowledge base), providing a specific action and target. However, it doesn't explicitly differentiate from sibling tools like 'get_document' (which likely retrieves a specific document) or 'query_knowledge_base' (which might search/filter documents), so it doesn't fully distinguish from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, exclusions, or comparisons to sibling tools like 'query_knowledge_base' (which might offer filtering) or 'get_document' (for single-document retrieval). It simply states what the tool does without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.