Server Quality Checklist

Tool Scores

• list_directoryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses a genuine side effect ('Resets idle timer') and the read-only nature of 'List'. However, it omits return format, recursion depth, and handling of hidden files, which are relevant behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: the primary action, path semantics, and a prerequisite plus side effect. No redundant words or filler, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-optional-parameter listing tool with no output schema, the description covers the essentials: what is listed, path relativity, and a required precursor. It lacks explicit mention of recursion or return structure, but these are minor given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage for the single 'path' parameter, with a description mirroring the tool's textual explanation. The description adds no extra parameter semantics beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List files and directories') and scope ('in the cloned repo') with a specific verb+resource structure. It distinguishes itself from sibling tools like read_file (which reads file content) and search_repo (which searches), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs 'Call ensure_repo first', providing a clear precondition for use. It also implies the tool is for initial exploration of the repo structure, naturally contrasting with alternatives like read_file or search_repo. However, it doesn't explicitly mention 'when not to use', so it stops short of a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It reveals that the path is relative to the repo root and that the tool "Resets idle timer," which is a non-obvious side effect. The read-only nature is implied by "Read" but not explicitly stated, which is acceptable for such a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four short sentences, each adding value: purpose, path constraint, prerequisite, and idle timer behavior. It is front-loaded and avoids unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 1-parameter tool with no output schema, the description covers the core purpose, path semantics, prerequisite, and a behavioral side effect. It would be slightly more complete if it explicitly stated that the file contents are returned, but this is strongly implied by the tool name and 'read' verb.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the single parameter 'path' with "Path relative to repo root," which is also repeated in the description. Since schema coverage is 100%, the description adds no additional semantic meaning. The baseline for high coverage is 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: "Read a file from the cloned repo," with a specific verb and resource. It distinguishes from siblings like list_directory and search_repo by focusing on reading a single file's content. The path constraint "relative to repo root" adds precise scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear prerequisite: "Call ensure_repo first," telling the agent when this tool can be used. It does not explicitly discuss alternatives or exclusions, but the context is clear enough for a simple read operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_repoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It adds useful context beyond the schema: the dependency on ensure_repo and the side effect of resetting the idle timer. However, it does not describe return format, error behavior if the repo isn't cloned, or permissions, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, all essential: purpose, optional parameters, prerequisite, and a behavioral note. It is front-loaded and contains no redundant or verbose language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 3 parameters and no output schema, the description covers the core purpose, parameters, and prerequisite. It could mention the return format or error behavior, but the given context (including sibling tools) makes the tool's role clear enough. It is complete relative to its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are already well-documented. The description adds marginal value by restating 'path (subdir)' and providing an example fileGlob, but it does not deepen understanding beyond the schema descriptions. This matches the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Search for text or regex in the cloned repo.' It also distinguishes itself from sibling tools like read_file and list_directory by focusing on content search across the repo, and mentions optional path/fileGlob to refine the search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'Call ensure_repo first,' providing a clear prerequisite and implied ordering. It also explains the optional parameters (path, fileGlob) and the side effect of resetting the idle timer. However, it does not explicitly mention when not to use this tool or name alternatives, though the sibling tools are distinct enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ensure_repoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description explains the core behavior (clone/reuse), the result (path and status), and a side effect (resets 5-minute idle timer). It covers the essential behavioral traits a caller needs, though it omits failure handling and auth details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with zero redundancy. The main action is first, followed by usage guidance and return/timer info. Every sentence provides distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple setup tool with a high-coverage schema, the description covers the purpose, usage order, return value, and side effect. No output schema exists, so the stated return of path and status suffices.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema (repoUrl and branch), giving 100% coverage. The description adds operational context but no additional parameter semantics, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb and resource: 'Clone a Git repo to a temp directory (or reuse existing clone).' It also distinguishes itself from siblings by noting it should be called first, making its role unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Call this first before search_repo, read_file, or list_directory,' providing direct usage ordering and naming specific sibling alternatives. This is a clear when-to-use statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repo_statusA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool has no cloning side effect and lists the returned fields (cloned state, path, URL, expiry). While it could mention potential errors or read-only behavior explicitly, the non-mutating nature is clearly implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the return information, and contains zero filler. Each word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter status tool with no output schema, the description covers purpose, return fields, and side-effect distinction. It is fully sufficient for an agent to understand when and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing to explain. Per rubric, 0 params receives a baseline of 4. The description adds no parameter info, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns repo status details (cloned state, path, URL, expiry) and explicitly differentiates itself from ensure_repo by noting it does not clone. This is a specific verb+resource pairing that eliminates ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Does not clone; call ensure_repo first if needed,' which is a clear when-not and points to the alternative tool. This provides strong usage guidance and prevents incorrect tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.