Server Quality Checklist

Tool Scores

• find_missing_docstringsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description gives no information about side effects, permissions, or whether the tool is read-only. The word 'Audit' implies non-destructive, but this is not explicit, and with no annotations, the burden falls entirely on the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, front-loaded with the verb 'Audit' and the core functionality. It is well-structured and free of unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description conveys the main function and outcome (coverage percentage), it lacks detail on the output format, how the audit is performed, and what 'include_private' entails. This leaves gaps in understanding, given the simple schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for parameters (target_dir, include_private). The description mentions 'directory' indirectly but does not explain either parameter in detail, especially include_private. The description fails to compensate for the lack of schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to audit Python files for missing docstrings and calculate coverage percentage. It specifies the verb 'Audit', the resource 'Python files', and the scope (modules, classes, functions), distinguishing it from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any conditions, use cases, or exclusions, offering no explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_file_astB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations (readOnlyHint, destructiveHint) are provided, so the description must carry the full burden of disclosing side effects. It states 'parse' which implies a read-only operation, but it does not explicitly state that the file is not modified, nor does it mention authentication, rate limits, or other behavioral aspects. The description is vague about non-obvious behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the action ('Parse a Python file') and then lists the elements of the returned outline. There is no unnecessary verbosity or repetition; every part contributes to the understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (single parameter, clear output), and an output schema exists (though not detailed here). The description sufficiently conveys what the tool does and what the outline contains, so the user can predict the result. It does not cover error conditions or edge cases, but these are not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter, 'file_path', with full coverage in the schema (name and type are provided). The description does not add extra meaning about the parameter, such as path format or constraints. Since schema coverage is high, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: parsing a Python file with AST and returning a detailed outline of its components (docstrings, imports, classes, functions, variables, line counts). It uses a specific verb ('parse') and resource ('Python file'), and distinguishes itself from sibling tools by offering a broad outline rather than focused analyses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not mention when to use this tool versus the sibling tools (e.g., find_class_methods, get_imports_graph). No explicit use cases, conditions, or alternatives are provided. Users must infer from the description that it is appropriate for a general overview.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_class_methodsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden and does clearly state what is inspected and returned. However, it does not disclose error behavior (e.g., class not found) or explicitly note that this is a read-only AST-level inspection, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: one purpose sentence followed by two param lines. Every sentence earns its place, with no filler or repetition of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter inspection tool with an output schema, the description adequately covers purpose and parameter meanings. It omits usage alternatives and failure behavior, but the output schema handles return-structure documentation, so this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description gives meaningful one-sentence docs for both parameters: file_path and class_name. This adds value beyond the raw schema, though it could have included more nuance such as path resolution or whether class_name needs to be fully qualified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Locate') and resource ('a specific class within a Python file'), then enumerates exact outputs: methods, argument signatures, type annotations, line numbers, and docstrings. This clearly distinguishes it from sibling tools like analyze_file_ast and find_symbol, which have broader or different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool instead of analyze_file_ast or find_symbol, and no exclusions or prerequisites are provided. The use case is only implied by the description of locating a class and retrieving its methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_imports_graphA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does state the core behavior and output types. However, it leaves important behavior unspecified, such as whether traversal is recursive, how non-Python or unreadable files are handled, and whether the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus a param docstring, front-loaded with the primary action and outputs. No redundant wording or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool, the basics are covered (input directory and output format), and an output schema exists to define return values. Yet it omits traversal scope and edge-case behavior, and the absence of annotations leaves safety/prerequisite details unexplained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, but the docstring provides direct semantics: target_dir is a path to a directory containing Python files. That fully compensates for the single parameter, though it does not add details like recursive traversal that might affect the parameter's meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb+resource: scan Python files in a directory to build an import dependency graph, and names two concrete outputs (structured map, Mermaid diagram). This clearly differentiates it from sibling tools that analyze individual symbols, files, or docstrings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The scope 'all Python files in a directory' implies when to use it, contrasting with per-file or symbol-level tools, but there is no explicit when-not-to-use statement or named alternative. It also does not mention prerequisites like valid directory permissions or expected input type beyond a path.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_symbolA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It conveys recursive traversal, accepts both file and directory paths, performs partial-name matching, and searches for four symbol categories. This meaningfully reveals behavior beyond the schema, though it does not explicitly mention side-effect safety or whether hidden files/symlinks are included.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with the core behavioral sentence appearing first. The parameter documentation adds value without redundancy, and every sentence contributes to understanding the tool's action or invocation parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a relatively simple two-parameter search tool, especially since an output schema is provided and return-value details need not be repeated. It lacks explicit alternative tool references, but given the output schema and simple parameter model, the description is largely complete for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema offers only parameter titles with 0% description coverage, so the description fully compensates. It clarifies that target_dir may be either a directory or file path, and that symbol_name can be a full name or partial fragment, giving the agent the exact operational meaning needed to invoke the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Recursively search') applied to a distinct resource ('directory or file') with a defined target ('classes, functions, methods, or variables'). It also identifies a key matching behavior (name or partial name), which sets it apart from sibling tools like find_class_methods or analyze_file_ast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use for finding symbols recursively across a directory or file, but it never explicitly states when to choose it over alternatives or when not to use it. The sibling tools are not mentioned, so the agent must infer the difference from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.