Server Quality Checklist

Tool Scores

• enable_docset_ftsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does (enable FTS) and the return value, but doesn't mention permission requirements, whether this is a one-time configuration or reversible, potential side effects, or performance implications. The description adds minimal behavioral context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with a clear purpose statement followed by structured Args and Returns sections. Every sentence serves a purpose, though the formatting with separate sections could be slightly more integrated for optimal front-loading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (which handles return value documentation) and only one parameter with good semantic coverage in the description, the description is reasonably complete. It covers the purpose, parameter meaning, and references sibling tools, though additional behavioral context would be beneficial for a configuration/mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides crucial semantic information about the single parameter: it specifies that 'identifier' refers to 'The docset identifier' and clarifies it should come 'from list_installed_docsets'. With 0% schema description coverage, this information significantly compensates for the schema's lack of documentation, though it doesn't provide format examples or validation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('enable full-text search') and resource ('for a specific docset'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools, but the specificity of enabling FTS for docsets distinguishes it from listing, loading, or searching documentation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by mentioning that the identifier should come 'from list_installed_docsets', suggesting a prerequisite workflow. However, it doesn't explicitly state when to use this tool versus alternatives or provide any exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentationA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: results are 'automatically truncated if they would exceed 25,000 tokens,' and it explains parameter defaults and constraints (e.g., 'max_results: 1-1000'). However, it doesn't cover aspects like rate limits, authentication needs, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose, followed by a structured list of parameters with clear explanations. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no annotations, but with an output schema), the description is mostly complete. It covers purpose, parameters, and key behavior (truncation), but lacks details on output format or error cases, though the output schema may mitigate this. For a search tool, it's reasonably comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the schema: it explains that 'docset_identifiers' are 'comma-separated' and sourced 'from list_installed_docsets', clarifies 'search_snippets' as 'whether to include snippets', and specifies 'max_results' range and default. This effectively documents all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for documentation across docset identifiers and snippets.' It specifies the verb ('search') and resource ('documentation'), but doesn't explicitly differentiate from sibling tools like 'load_documentation_page' or 'list_installed_docsets' beyond mentioning docset identifiers come from the latter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through the mention of 'docset identifiers (from list_installed_docsets)' and the parameter descriptions, but lacks explicit guidance on when to use this tool versus alternatives like 'load_documentation_page' or 'enable_docset_fts'. No exclusions or clear context for tool selection are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_installed_docsetsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and adds valuable behavioral context: it discloses that results are automatically truncated if exceeding 25,000 tokens and specifies the return condition for no installed docsets. This goes beyond basic functionality, though it could mention performance or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by two concise sentences adding critical behavioral details. Every sentence earns its place with no waste, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema exists), the description is largely complete. It covers purpose, edge cases (empty list), and a key limitation (token truncation). With an output schema, it need not explain return values, but could slightly enhance by mentioning format or ordering.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents inputs. The description appropriately does not add parameter details, maintaining focus on behavior. Baseline is 4 for zero parameters, as it avoids unnecessary repetition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all installed documentation sets in Dash') and resource ('documentation sets'), distinguishing it from siblings like search_documentation or load_documentation_page. It provides precise scope by specifying 'installed' and the context of Dash.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving installed docsets, but does not explicitly state when to use this tool versus alternatives like search_documentation. It mentions an empty list result condition, which gives some context, but lacks explicit guidance on scenarios or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_documentation_pageA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adequately describes the core operation (loading content) and output format ('plain text with markdown-style links'), but lacks details about error handling, performance characteristics, or authentication requirements. The description doesn't contradict any annotations (since none exist), but could provide more complete behavioral context for a tool that presumably makes network requests.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise, with three focused sentences that each earn their place: the purpose statement, parameter guidance, and return value description. It's front-loaded with the core functionality and wastes no words while covering all essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (network request to load documentation), no annotations, and the presence of an output schema (which handles return value documentation), the description is mostly complete. It covers purpose, usage context, parameter semantics, and output format. The main gap is lack of error handling or performance details, but with an output schema handling return values, this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that the load_url parameter must come from 'search_documentation' results and must point to 'the local Dash API at 127.0.0.1', providing crucial context about parameter provenance and format that the schema alone doesn't capture. For a single parameter with no schema descriptions, this represents good compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Load a documentation page') and resources ('from a load_url returned by search_documentation'), distinguishing it from sibling tools like search_documentation (which finds pages) and list_installed_docsets (which lists available documentation sets). The description explicitly references the sibling tool search_documentation, establishing a clear workflow relationship.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('from a load_url returned by search_documentation') and includes important prerequisites about the URL format ('must point to the local Dash API at 127.0.0.1'). This creates clear context for usage and distinguishes it from alternative approaches that might involve different URL sources or direct file loading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.