Server Quality Checklist

Tool Scores

• get_file_metadataB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the purpose and hints at performance ('without reading full contents') but does not specify what metadata fields are returned, error handling for missing files, or permission requirements. This is insufficient for a tool with no annotation safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the action and scope. It contains no redundant information and is efficiently concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple interface and an output schema, which covers return values. However, the description lacks parameter semantics and explicit usage context, and with no annotations, it leaves gaps around error behavior and when to prefer this over get_repository_tree. It is slightly below the minimum viable completeness for a standalone tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining the parameters. It does not clarify the format of 'relative_path' or what 'workspace' refers to exactly. The parameter names are somewhat self-explanatory but lack necessary detail about path conventions and options.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and clearly identifies the resource ('metadata for one repository file'). It distinguishes itself from sibling tools like read_file and read_file_range by explicitly stating 'without reading its full contents'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without reading its full contents' implies a lightweight metadata use case, but it does not explicitly name alternatives or state when not to use this tool. Sibling tools like read_file are mentioned indirectly but not directly contrasted, making the usage guidance implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_filesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does not mention search recursion, case sensitivity, hidden files, how limit affects results, or whether it respects ignore files. This leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short, front-loaded sentences. The first states the core action, and the second adds a useful use-case hint. There is no waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema covers return values, reducing the burden there. However, the description lacks parameter semantics and behavioral details, leaving gaps for an agent to correctly invoke the tool. It is minimally adequate for a simple tool but incomplete in important areas.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only mentions 'glob pattern' without explaining the expected syntax, the meaning of workspace, or the role of limit. It does not compensate for the missing schema parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific action: 'Find repository files by glob pattern.' It distinguishes itself from sibling tools like get_repository_tree or read_file by focusing on pattern-based file discovery rather than tree navigation or content reading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use the tool: 'Use this when filenames, extensions or approximate locations are known.' This provides clear context, though it does not explicitly name alternatives or define when NOT to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repository_treeA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It conveys the output is a bounded directory/file map and clarifies it does not retrieve contents. However, it does not disclose error behavior, whether it follows nested submodules, or any specific limitations beyond 'bounded', leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose in the first and usage guidance in the second. Every word earns its place; no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The purpose and primary usage are clear, and the output schema exists to document return values. However, the lack of parameter explanations and minimal behavioral disclosure make it adequate but incomplete for a tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description does not explain workspace, max_depth, or max_entries. The only hints are 'one repository' and 'bounded', which are indirect and insufficient. The description fails to compensate for the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes a specific verb and resource: 'Return a bounded directory and file map for one repository.' It clearly differentiates from siblings by focusing on structure rather than contents, with the explicit exclusion 'Do not use it to retrieve file contents'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: 'Use this to understand repository structure before selecting files.' It also gives an explicit when-not: 'Do not use it to retrieve file contents.' However, it does not name an alternative tool like read_file, so it stops short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses important behavioral traits: the tool only reads UTF-8 text, enforces a size limit, and will reject files that are large, binary, denied, or outside the workspace. This goes beyond the schema and gives agents a clear picture of operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the core purpose, then lists rejection conditions. Every word adds value with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the essential operational envelope: bounded reading, text-only, and rejection conditions. The output schema exists, so return values are documented elsewhere. Minor gaps remain (e.g., what constitutes 'large' or how 'bounded' is determined), but overall it is sufficiently complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explicitly explain any parameter. It vaguely aligns 'bounded amount' with max_chars and mentions workspace/relative path via rejection of out-of-workspace files, but does not clarify required syntax, default behavior, or how max_chars works. The description provides minimal compensation for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (read), the resource (one UTF-8 text file), and a key constraint (bounded amount). It also lists rejection criteria (large, binary, denied, out-of-workspace) which further defines the tool's scope and distinguishes it from sibling tools like read_file_range and get_file_metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: read text files up to a bounded size. The rejection of large, binary, denied, and out-of-workspace files gives context about when not to use it. However, it does not explicitly name alternatives or provide when-to-use/when-not-to-use guidance compared to sibling tools like read_file_range or get_file_metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_file_rangeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description provides some behavioral context (one-indexed, UTF-8) but does not disclose edge-case behavior like invalid line ranges, out-of-bounds, or error handling. For a read operation this is acceptable but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundant information. The first sentence front-loads the core action and resource, the second adds a usage guideline. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with four self-explanatory parameters and an output schema available. The description covers the essential nuance (one-indexed range, preference over whole-file read), making it sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning to start_line and end_line by specifying one-indexed, and to relative_path by specifying UTF-8 text file. However, it does not elaborate on workspace or path resolution beyond what parameter names imply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a one-indexed line range from a UTF-8 text file, distinguishing it from sibling read_file by focusing on a specific range rather than the whole file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends using this tool over reading a whole file when the relevant location is known, naming the alternative and giving a clear condition for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_workspacesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'List' implies a read-only operation, and 'currently available for inspection' adds context about the dynamic nature of available repositories. It could explicitly mention safety, but the read-only nature is clear enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the action and resource. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema (present), the description is complete. It provides the purpose, usage timing, and context, and sibling tools give additional framing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is effectively 100%. The description adds no parameter details, but none are needed. The baseline for zero-parameter tools is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' with a clear resource ('local repositories currently available for inspection'). It distinguishes itself from sibling repository-specific tools by focusing on the workspace enumeration step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Call this before repository-specific tools when the workspace name is unknown.' This provides direct guidance on prerequisites and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.