Server Quality Checklist

Tool Scores

• create_docB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden, but it only states the basic creation action and optional content. It does not disclose what happens if the file already exists (overwrite vs. error), permission requirements, or side effects. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no redundant words, clearly front-loading the primary action. It is clean and scannable, though it sacrifices some depth for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple create operation, but the absence of annotations and output schema places a heavy burden on the description. It fails to address important operational realities such as slug conflicts, overwrite behavior, return values, or differentiation from write_doc. This leaves notable gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all parameters with descriptions and an enum, so the baseline is 3. The description adds only a minor interpretive note about docType choosing categories and 'optional initial content,' but provides no examples or extra syntax details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action ('Create a new documentation file') and distinguishes this tool from siblings by emphasizing 'new' and optional initial content. It also mentions docType for category selection, which adds purpose specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives such as write_doc. It does not mention typical scenarios, exclusions, or what to do with an existing slug. The instruction to use docType is about parameter selection rather than usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_docA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, but it only states the obvious read action. It does not disclose edge cases (e.g., missing file), return format, or confirm that no modifications occur—traits an agent might need to know.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the action and resource, and no unnecessary words. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with only 2 parameters, the description is complete enough to convey what it does and how to identify the file. It lacks return-value expectations or error behavior, but these are not critical for a straightforward read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description's 'slug (filename without path)' essentially mirrors the schema's 'Filename or slug' description. No significant additional meaning is added beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Purpose is clear: 'Read the content' specifies the exact action on a documentation file, and the parameters (type and slug) are stated. This distinguishes it from siblings like list_docs, write_doc, and delete_doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you need the content of a specific doc) but provides no explicit guidance on when to choose this over alternatives like search_docs or list_docs. There are no exclusions or alternative mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_docA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It explicitly states 'overwrite' (destructive behavior) and 'Creates parent folder if needed' (side effect), plus the markdown requirement. This is meaningful behavioral disclosure, though it doesn't mention return values or auth requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences front-load the main action and constraints. Every piece of information earns its place; no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write tool, the description covers core behavior (write/overwrite, content type, folder creation). It doesn't address sibling differentiation (create_doc) or return values, but the schema covers params and the behavior is adequately described for a low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions, so the baseline is 3. The description adds minimal parameter insight—'Content must be markdown' echoes the schema's 'Markdown content to write' and 'Creates parent folder if needed' is more about folder side effects than parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Write or overwrite') and resource ('documentation file'), and notes constraints. However, it doesn't differentiate from sibling tool 'create_doc', so it is clear but lacks sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to choose this tool over create_doc or other siblings. No exclusions or prerequisites are mentioned beyond the markdown constraint, so the agent gets no help choosing between write and create alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_docA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The verb 'Delete' implies a destructive action, but the description does not state that the deletion is permanent, irreversible, or what exactly happens to the file (e.g., removed from storage). This lack of warning is a significant gap for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single direct sentence with no filler. Every word is essential: it names the action, target, and the two parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete operation with complete schema coverage and no output schema, the description conveys the core operation. However, it does not provide details about side effects, prerequisites (e.g., whether the file must exist), or the irreversible nature of deletion. Given the lack of annotations, the description is adequate but not fully complete for safe autonomous use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both docType and slug with 100% coverage, including the enum values for docType. The description simply repeats the parameter names ('by type and slug') without adding any new semantic meaning, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action (delete), the resource (documentation file), and the required identifiers (type and slug). This clearly distinguishes it from sibling tools like read_doc and write_doc by the delete action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly signals when to use the tool: when a documentation file needs to be deleted by type and slug. It does not explicitly mention when not to use it or compare with alternatives, but the purpose is unambiguous and the context is clear enough for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that keyword search is case-insensitive and that semantic search depends on local embeddings (DOCS_EMBEDDING_PROVIDER=local), which is a behavioral trait. However, it does not disclose return format, pagination, or whether the operation is read-only, leaving gaps for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the purpose, and every sentence adds value. It avoids redundancy and is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite four parameters and no output schema, the description covers the main behavioral context (mode switches) but omits return value details and error conditions. Given the tool's simplicity, this is adequate but leaves gaps typical of an undisclosed output format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds a small behavioral detail (case-insensitivity) and restates the mode default already present in the schema, but does not significantly deepen understanding of parameters beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches documentation by text, using the verb 'search' and a specific resource ('documentation'). It also distinguishes itself from sibling tools (list_docs, read_doc, write_doc, etc.) by its search functionality, and the mode detail (semantic vs. keyword) adds specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use semantic vs. keyword search based on the environment variable, which is helpful for mode selection. However, it does not explicitly state when to use this tool over alternatives (e.g., list_docs or read_doc), relegating tool selection to implication rather than explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_docsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It communicates a read-only listing operation and the filter, but does not disclose return format (e.g., whether it returns full doc contents or metadata), sorting, pagination, or potential error cases. This is adequate for a simple list but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, immediately conveys the action and filter, and contains no filler. It is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single optional parameter and no output schema, the description covers the core purpose and filtering. However, the lack of return value explanation and absence of guidance vs. search_docs leaves some contextual gaps, though minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents the 'docType' parameter with enum values and a description. The tool's description repeats the enum list but adds no additional semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('List') and resource ('documentation files'), and the optional filter by doc type adds a distinct scope. It differentiates from siblings like read_doc and write_doc, which imply targeting specific docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for enumerating documentation files, optionally filtered by type. However, it does not explicitly state when to prefer this over search_docs or how it differs from reading a single doc. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• product_vision_and_strategyA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly discloses the key side effect: if the file does not exist, it is created with a default template. This is important context for the agent, though it omits details like return value or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the primary action and then a conditional side effect. Every sentence earns its place, with no redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description adequately conveys the main behavior and the side effect. It does not explain the return format, but this is largely inferable from the 'View' action and the simple nature of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema fully covers parameter semantics. Per the rubric, the description does not need to add parameter information; the baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('View') and a specific resource (the product vision and strategy document), which clearly distinguishes it from generic document tools like read_doc or write_doc. However, the second sentence introduces a creation side effect, slightly blurring whether this is purely a read operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you need the product vision and strategy document, including automatic initialization if missing. It does not explicitly compare with sibling tools or state when not to use it, so guidance is only implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.