Server Quality Checklist

Tool Scores

• analyze_sphinx_structureB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only says 'Analyze the structure'. It does not disclose side effects, return format, or the scope of analysis, which is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, front-loaded sentence with no unnecessary words. The description is appropriately concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description should explain what 'structure' analysis returns, but it does not. The tool has only two well-documented parameters, but completeness is undermined by missing output/behavior details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for sourcePath and depth. The description itself adds no parameter detail beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'Analyze' with resource 'structure of a Sphinx documentation project', clearly distinguishing it from sibling convert tools. It states exactly what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for analyzing project structure but provides no explicit when-to-use or when-not-to-use guidance relative to sibling convert tools. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_sphinx_directoryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only states 'Convert' without explaining whether the operation is read-only, whether it modifies the source files, what happens to existing output files, or how chunking and optimization affect the process. The lack of any side-effect or safety information leaves the agent under-informed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, directly stated sentence. It front-loads the core action, resource, and result with no filler or redundant information. Every word contributes value, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (directory processing, multiple options, no output schema, no annotations), the description is too brief. It does not mention what the output looks like, how the conversion works, whether it's safe, or what 'LLM-optimized' entails. The agent would need to rely solely on the schema, which is insufficient for understanding the tool's full behavior and implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%, with sourcePath and outputPath described in the schema. The description adds no additional parameter semantics, and the 'options' parameter lacks a top-level schema description. Since coverage is in the moderate range, the description does not need to fully compensate but could have added clarity about options like recursive or preserveStructure, which it fails to do.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Convert an entire Sphinx documentation directory to LLM-optimized Markdown' clearly specifies the verb (Convert), the resource (Sphinx documentation directory), and the output (LLM-optimized Markdown). It distinguishes from the sibling tool 'convert_sphinx_file' by explicitly targeting directories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for whole directories, but it does not explicitly state when to use this tool versus 'convert_sphinx_file' or 'analyze_sphinx_structure'. No exclusions or alternative recommendations are provided. The usage context is clear from 'entire directory' but lacks explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_sphinx_fileA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the basic conversion action but does not explain side effects (e.g., whether outputPath is overwritten), error conditions, or what 'LLM-optimized' entails. This leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately states the tool's purpose. There is zero fluff or redundancy, and the key scoping term ('single') is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a nested options object, no output schema, and no annotations. The description is too minimal to convey what the tool returns, how options like chunkSize and optimize affect output, or any side effects. This lack of completeness makes it hard for an agent to predict behavior for non-trivial conversions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter information at all. Schema coverage is 67% (sourcePath and outputPath have descriptions, but options object lacks a description). Since coverage is not high and the description does not compensate, it provides little value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Convert') and resource ('a single Sphinx RST file') with a clear output ('LLM-optimized Markdown'). The word 'single' distinguishes it from the sibling tool convert_sphinx_directory, making it clear this is for individual files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies usage for a single file via 'single', providing context that this is for one RST file at a time. However, it does not explicitly mention alternatives like convert_sphinx_directory or exclusions, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.