Server Quality Checklist

Tool Scores

• markdown_outlineA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds critical context such as path requirements (absolute, no URL-encoding, must exist) and output format defaults. While it doesn't mention error handling or side effects, the read-only nature is implied and the provided constraints are valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with separate sections for the core one-liner, usage guidelines, path requirements, example paths, args, and returns. It is somewhat lengthy but every section adds needed information; the example paths are a minor addition but not wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, usage boundaries, parameter semantics, path constraints, and return format. It doesn't detail the exact structure of the returned outline, but the 'Returns' line gives a minimal overview. For a relatively simple file-reading tool, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates fully with an Args section that explains each parameter: paths (list of absolute Markdown paths) and output_format (valid values 'json' or 'markdown', default behavior). This goes well beyond the bare schema, including allowed values and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns an outline for each Markdown file, listing headings, levels, and line numbers. It uses a specific verb and resource, and the inclusion of 'Markdown' distinguishes it from sibling tools like python_outline and dir_tree.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit agent usage guidelines are provided, stating when to use the tool (extract/display Markdown structure, navigation, summary, documentation analysis) and when not to use it (non-Markdown files or reading full content). This gives clear exclusions and context without explicitly naming alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openapi_get_operation_detailsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses important behavioral traits: path must be absolute, unencoded, exist, and be a valid OpenAPI file; expand_refs behavior for resolving refs; format_output options; and return structure including error handling. This is more than enough for a read-only file inspection tool, though it does not elaborate on edge cases like empty results or permission issues.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose, usage guidelines, path requirements, args, returns. It is substantial yet every sentence provides necessary information for correct use. The front-loaded purpose and usage guidelines make it easy to scan, and the detailed parameter explanations are justified given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all essential aspects: what it does, when to use it, path constraints, parameter semantics, and return format (including error field). Despite having an output schema, the description still fully explains the return values and error handling, making it self-contained. No important context for selecting or invoking this tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so comprehensively: spec_path is defined as the absolute path, selectors are explained with examples, expand_refs has a clear default and effect, and format_output options are listed. This adds significant meaning beyond the raw schema, which only lists types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get detailed information for specific OpenAPI operations.' This is a specific verb+resource pair that distinguishes it from siblings like openapi_list_operations, which presumably lists operations. The mention of selectors (operationId, method+path, path) further clarifies its targeting of specific operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Agent usage guidelines' section explicitly states when to use the tool: 'Use this tool when you need detailed information about specific API operations.' It explains how to use selectors and adjust expand_refs/format_output. However, it does not explicitly mention when not to use it or directly contrast with sibling tools like openapi_list_operations, so it lacks exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dir_treeA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the depth limitation, output format options, and path requirements (absolute, no URL-encoding, must exist). It also notes that an error dict is returned on failure, which is transparent. However, it doesn't explicitly state that this is a read-only operation with no side effects, though that is implied by the nature of the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (description, usage guidelines, path requirements, example paths, args, returns). Each sentence adds value and there is no irrelevant information. It is appropriately sized for a tool with multiple parameters and usage constraints, front-loading the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 params, one required) and the presence of an output schema, the description is complete. It covers purpose, usage guidelines, path constraints, parameter semantics, and return behavior. The mention of error handling and output formats rounds out the context, so agents have all necessary information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates thoroughly with an Args section explaining each parameter: root_path (absolute path), max_depth (nesting depth, default 1), and output_format (json/markdown, default derived from server). It provides example paths for both Windows and Linux, adding meaning beyond the schema's basic type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a compact file and folder tree with depth limitation. It specifies the resource (file/folder tree) and distinguishes it from siblings like python_outline and markdown_outline, which focus on source code outlines rather than directory structure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit agent usage guidelines: when to use (quick overview, hierarchy analysis) and when not to use (reading file contents, non-existent/relative paths). This covers both positive use cases and exclusions, making it clear how to select this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• python_outlineA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the read-only nature (returns an outline), the output format options, path constraints, and that it does not read file contents in detail. However, it does not describe error behavior for invalid paths, though it states paths must exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Agent usage guidelines, Path requirements, Args, Returns). It is front-loaded with the main purpose and every sentence adds value, including examples and constraints. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and an output schema, the description is complete. It covers purpose, usage context, parameter semantics, path prerequisites, and return format. The presence of an output schema reduces the need to explain return structure, and the description covers all other aspects adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates fully. It explains both parameters: 'paths' as a list of absolute paths, and 'output_format' with allowed values ('json' or 'markdown') and default behavior. It even provides example paths for Windows and Linux.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Returns an outline for each Python file: imports, classes, functions, docstrings' with a specific verb and resource. It clearly distinguishes itself from siblings like dir_tree by focusing on Python file structure rather than directory trees.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this tool when you need to understand the structure of Python code files' and 'Do not use for non-Python files or for reading file contents in detail.' Path requirements and examples are also given, fully explaining when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openapi_list_operationsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It thoroughly describes behavior: filtering, pagination, output formats, return structure (operations, count, total_count, error), and path requirements (absolute, no URL-encoding, must exist and be valid). This goes well beyond a simple summary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though the description is long, it is well-structured with sections (Agent usage guidelines, Path requirements, Args, Examples, Returns). Each section provides essential information without redundancy. The main purpose is front-loaded, and all content earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 7 parameters, no annotations, and no output schema. The description covers all aspects comprehensively: purpose, usage, parameter semantics, examples, return format, and edge cases. It is fully self-contained for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates with an Args section that explains every parameter in plain language, including defaults and example values. The examples further illustrate usage with realistic JSON inputs, making the parameter semantics completely clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'List operations from an OpenAPI specification file', which clearly specifies the verb (list) and resource (operations in an OpenAPI spec). It distinguishes itself from siblings by explicitly noting it's for exploring endpoints, methods, and summaries without detailed schemas, which contrasts with sibling 'openapi_get_operation_details'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance with bullet points: 'Use this tool when...' and 'Do not use for...'. It clearly states the scenarios for use (exploring available operations) and the exclusion (detailed parameter/response information), which implicitly points to the sibling getting details tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.