Server Quality Checklist

Tool Scores

• statsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It states the tool provides statistics but does not explicitly confirm it is read-only or has no side effects. For a stats tool this is likely non-destructive, but the description leaves that unspoken, which is a gap given the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the purpose and lists specific statistics. There is no redundancy or unnecessary detail, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with an output schema, the description adequately summarizes what the tool does. Mentioning document count, tags, and index age is enough for an agent to decide when to call it. The output schema supplies return details, so omitting return format isn't a deficiency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100% and the description has no need to add parameter details. The baseline of 4 applies, and there is nothing about the inputs that the description fails to cover.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides statistics about the knowledge base, enumerating specific data points (document count, tags, index age). This makes the purpose unambiguous and distinguishes it from sibling tools like search_knowledge_base (which returns matches) and list_documents (which returns items), though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus siblings. It doesn't mention that it's for aggregate summaries rather than per-item retrieval, when to prefer it over listing documents, or any exclusions. The only implied usage is 'when you want overall statistics,' but this is not made explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the behavioral burden. It clearly conveys that this is a listing operation and that filters can be applied, but doesn't mention matching semantics (exact match vs partial), whether the result is read-only, or what happens when no docs match. Since output schema covers return structure, these gaps are moderate, not critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the primary purpose and adds the key condition ('optionally filtered') without any redundancy. Every word contributes to understanding the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple list with two optional parameters and an output schema available, so the description is largely adequate. The missing guidance on when to choose search_knowledge_base over this tool leaves only a minor gap in full context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both parameters already have clear inline descriptions in the schema. The description text independently mentions only generic optional filtering, so it does not improve on the structured schema data, keeping this at the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the exact action ('List') and resource ('every article in the knowledge base'), setting it apart from siblings like search_knowledge_base, which implies a searching use case. The optional filtering is mentioned, so an agent can recognize this as the enumeration tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies 'use this to list documents' but gives no explicit direction about when to prefer this over search_knowledge_base, or when not to use it. It relies on the agent inferring the distinction from the '*' keyword, which is plausible but not directly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tagsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does disclose that the output includes document counts, which is a behavioral trait. However, it doesn't specify pagination, ordering, or whether hidden/empty tags are excluded, which would be useful for an agent calling the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that conveys the purpose precisely without any fluff. It is front-loaded with the action and resource, making it easy to parse. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameter-less tool with no output schema details, the description is largely sufficient. The main minor gap is not disclosing whether the tags list is flat or sorted, or if it includes counts for documents the agent doesn't have access to. However, these are edge cases for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no schema to elaborate on. The description appropriately explains what the output contains ('document counts'), providing value beyond the empty schema. A baseline of 4 is given because no parameters exist to complicate invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all tags used across the knowledge base, with document counts' is clear and specific regarding verb ('List'), resource ('tags'), scope ('across the knowledge base'), and additional detail ('with document counts'). Among its siblings, it clearly differentiates itself from tools like 'get_document' or 'search_knowledge_base'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when listing tags with counts is needed) but provides no explicit guidance on when not to use it or which alternative to choose. For instance, it doesn't state that to get documents you'd use 'list_documents' or to search use 'search_knowledge_base'. This leaves some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_knowledge_baseA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure burden. It states the tool accepts natural language or keywords, which is a behavioral trait about query interpretation. It implies a read-only search (no side effects mentioned), and does not introduce any misleading or surprising behavior. The presence of an output schema further clarifies return format, so a 4 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste. The core function is front-loaded, and the usage guidance immediately follows. Every sentence earns its place, making it an example of ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter search tool with an output schema, the description is complete. It tells the agent when to use it and provides a clear scope. It does not describe return format or pagination, but the output schema covers that. No critical missing context is apparent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: query, tag, and limit all have descriptive text in the schema. The description adds no additional parameter-specific meaning beyond what the schema already provides. Per guidelines, baseline is 3 when schema carries the burden, and the description does not enhance it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the company knowledge base with natural language or keywords, specifying the verb and resource. It does not explicitly differentiate from sibling tools like list_documents or get_document, but the verb 'search' inherently implies a query-based retrieval distinct from listing or fetching, so purpose is clear though sibling differentiation is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this FIRST whenever a question might be answered by internal documentation.' It also provides examples of content (processes, runbooks, policies). It does not mention alternatives or exclusions, but the 'FIRST' directive and clear context make this strong guidance for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does a reasonable job: 'Fetch' indicates a read-only operation, and 'full Markdown content' specifies the response format. It does not mention error/failure behavior or whether auth is required, but for a simple one-id getter this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that says exactly what the tool does with no filler. It earns its place and leaves nothing to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter fetch tool with a rich parameter description and an output schema, the description is complete enough: it identifies the resource, output format, and article scope. Explicit sibling routing would help but is not necessary for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the doc_id parameter is well documented with its source and an example. The tool description itself adds no parameter-level information, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Fetch') and resource ('knowledge-base article'), and 'full Markdown content' clearly distinguishes it from sibling tools like search_knowledge_base or list_documents. An agent can infer the exact function without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than explicit: fetch full content when you need an article's Markdown, and the doc_id parameter description says the id comes from search_knowledge_base or list_documents. However, the description itself does not state when to prefer this over alternatives or what prerequisites exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reload_knowledge_baseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the core behavior (re-scanning the directory) and the return value (number of indexed documents), which is good. However, it does not disclose potential side effects, such as whether the operation is expensive, whether it also removes entries for deleted files, or whether any special permissions are required. This leaves some ambiguity about the tool's full impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. The first sentence front-loads the action and purpose; the second explicitly states the return value. Every word earns its place, and the structure is compact and direct.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with zero parameters, an empty input schema, and an output schema that documents the return format. The description covers the tool's purpose and return value, which is sufficient for an agent to call it correctly. A minor gap is not stating whether it handles removals, but this is not essential for the primary use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, and the input schema is empty with 100% schema description coverage. With no parameters to document, the description does not need to add any parameter details. The baseline for zero-parameter tools is 4, and nothing here lowers that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Re-scan' with a clear resource, 'the knowledge-base directory', and states the intended effect: 'to pick up new or edited articles'. This clearly distinguishes it from sibling tools like search_knowledge_base, get_document, list_documents, list_tags, and stats, which are all read/search operations. The tool's purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'to pick up new or edited articles' conveys the primary use case: call this tool after adding or modifying articles to refresh the index. However, it does not explicitly state when not to use it or name alternatives, though no direct alternative exists among the siblings. The context is clear enough for an agent to infer when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.