Server Quality Checklist

Tool Scores

• analyze_projectA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral context beyond the annotations by explaining that the tool works with 'bounded local evidence' and produces confidence levels, evidence paths, readiness, and safety details. This is consistent with readOnlyHint=true and provides useful insight into the tool's operation and outputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that efficiently conveys the tool's purpose and key output aspects without unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with one parameter and strong annotations, the description is fairly complete: it explains the inference scope and the types of outputs (confidence, evidence paths, readiness, safety details). However, it lacks explicit usage guidance relative to sibling tools, which is a notable gap given there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the single parameter 'path' with 100% coverage, including its meaning ('Absolute path inside an approved root'). The description does not add further parameter-specific detail, so it meets the baseline for schema-dominated parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Infer' and identifies the resource as 'project purpose and capabilities' with a clear scope ('from bounded local evidence'). It also lists the output dimensions (confidence, evidence paths, readiness, safety details), which distinguishes it from sibling tools like summarize_project or read_project_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use analyze_project versus its siblings. It does not mention alternatives, exclusions, or prerequisites beyond what the schema already states about the path. Usage must be inferred from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• summarize_projectA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, which is repeated in the description. The description adds the term 'bounded' (suggesting output limits) and 'structural summary' (indicating the nature of the result), but it does not disclose behavior for invalid paths or detailed output structure. This is slightly more than annotation repetition, but still limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently states the tool's purpose and constraints. Every word adds value with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description adequately conveys purpose and scope. The term 'structural summary' is somewhat vague, and no output format is described, but given the tool's simplicity, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single 'path' parameter (100% coverage), and the description does not add additional semantic details beyond what is in the schema. The baseline of 3 applies because schema covers the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Return), the resource (a directory), and the outcome (a structural summary). It distinguishes itself from siblings like read_project_file (which reads files) and analyze_project (which likely performs deeper analysis) by focusing on a bounded, structural summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this to get a structural summary of a directory within an approved root. However, it does not explicitly mention when to use this tool instead of alternatives like analyze_project, nor does it state exclusions. The usage guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_project_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, but the description adds important behavioral context: sensitive-path filtering and redaction. This discloses that output may be filtered or redacted, which is valuable beyond the structured annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action and constraining criteria. Every phrase (small, recognized, inside approved root, filtering, redaction) earns its place, making it compact and information-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read-only tool with annotations and no output schema, the description covers the essential purpose and key behavioral qualifiers. It could be more explicit about what counts as 'recognized' or the size limit, but given the tool's simplicity and existing annotations, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the single parameter 'path' as 'Absolute path inside an approved root' (100% coverage). The description adds context that the path refers to a small recognized text file, but does not provide additional syntax or format details, so it meets the baseline for well-covered schemas without going beyond.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Read'), the target ('small recognized text file'), and the scope ('inside an approved root'), with additional qualifiers ('sensitive-path filtering and redaction') that distinguish it from sibling tools like analyze_project or summarize_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies appropriate use for reading small recognized text files within approved roots, and the readOnlyHint annotation confirms it's a safe read operation. However, it does not explicitly state when to prefer this tool over siblings or provide exclusions, so it provides clear context but no explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inventory_rootsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, and the description adds the meaningful constraint 'Does not scan file contents,' which clarifies that the tool only returns directory paths and does not inspect contents. This goes beyond the annotation's read-only flag by describing operational scope, adding genuine value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and directly conveys the tool's purpose and limitation. Every word serves a distinct function: it states the action ('List'), the object ('directories'), the scope ('explicitly configured'), and the non-behavior ('Does not scan file contents'). No waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity—no parameters, no output schema, and a single clear function—the description fully covers what an agent needs: what the tool returns (root directories) and what it does not do (scan contents). The sibling tools suggest broader analysis, so this narrow scope is appropriately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter burden on the description. The baseline for no parameters is 4, and the description does not need to add extra parameter details. It correctly focuses on the tool's behavior rather than parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and clearly identifies the resource ('explicitly configured directories this server is allowed to inspect'). It distinguishes itself from sibling tools like summarize_project and analyze_project by focusing on allowed inspection roots rather than content analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: to list approved root directories. It also notes the limitation 'Does not scan file contents,' implying use only when directory names are needed, not file-level inspection. However, it does not explicitly name alternatives or exclude specific use cases, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.