Server Quality Checklist

Tool Scores

• remove_projectB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only says 'removes' and does not clarify whether this is permanent, whether it deletes associated files, or what happens to the project's data. The word 'registered' provides a hint that it may only remove a registration, but this is not elaborated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that delivers the core purpose without any wasted words. It is front-loaded and easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is too sparse. It does not mention side effects, error conditions, reversibility, or any safety context. The simplicity of the tool does not excuse the lack of behavioral detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema includes a description for the single 'name' parameter, covering 100% of parameters. The tool description adds no additional meaning beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Removes the registered project' clearly states the verb and resource, and it is distinct from sibling tools like add_project and list_projects. It immediately conveys what the tool does without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when removing a project, but it provides no explicit guidance on when to use it versus other tools, nor does it mention prerequisites or alternative approaches. The purpose alone suggests the usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the use of ripgrep and the ability to limit search scope, adding value beyond the raw schema. However, it does not mention output format, read-only nature, or any limitations. With no annotations provided, the description carries the full burden and only partially fulfills it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and effective: two sentences that state purpose, capabilities, and a usage tip. It is front-loaded with the main action, with no unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description should explain what the tool returns or any important behavior. It covers the core search operation and constraints, but omits return value details (e.g., file paths, snippets) and potential edge cases, leaving the agent with incomplete information for full usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds a practical tip about Korean keywords, but does not significantly elaborate on parameter formats or provide examples beyond what the schema already includes. It reinforces optionality of project and file_pattern but adds little new semantic meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches code by keyword in registered projects, with specific capabilities like project restriction and file pattern filtering. This distinguishes it from sibling tools like read_file or list_projects, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for keyword-based code search in registered projects, but does not explicitly contrast with alternatives like get_file_tree or read_file. It provides context (search scope, file patterns) but lacks explicit when-to-use or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_projectA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It provides a meaningful post-condition ('registered projects can be searched') and an implicit prerequisite ('Git project at local path'). However, it omits details like idempotency, duplicate handling, or whether the path is validated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary action and followed by context about post-registration usage. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple registration tool with a fully documented schema and no output schema, the description gives enough context: what it does and what it enables. It could mention errors or return values, but these are not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with clear descriptions (path, name, description). The tool description does not add extra parameter-level meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('register a Git project at a local path') and its resource, and immediately distinguishes it from search-oriented siblings by noting that registered projects become searchable via search_code, get_file_tree, and read_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: before using the search tools on a project. It clearly positions add_project as a prerequisite for those tools, though it does not explicitly mention alternatives like remove_project or list_projects.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_treeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the core behavior (returns file/folder structure) and implies a safe read operation, but does not describe return format, depth limits beyond schema, or any side effects. For a simple read tool this is acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the core function and followed by usage context. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with two parameters and no output schema, the description sufficiently covers purpose and usage. It lacks an explicit description of the return format, but the concept of a file tree is intuitive and the sibling tools provide context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with descriptions for both 'project' and 'depth' including the default value. The description adds no additional parameter semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Returns the file/folder structure of the project' with a specific verb and resource, distinguishing it from siblings like search_code and read_file. The second sentence adds a use case ('when narrowing search scope'), which reinforces its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear usage context: 'used when understanding the project structure or narrowing down the search scope.' This implies a contrast with search tools, though no sibling tool is explicitly named or excluded.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of conveying behavioral traits. It implies a read-only operation via 'returns' but does not explicitly state that it has no side effects, requires no permissions, or does not modify data. It also does not disclose any limitations like pagination or ordering. This is acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences. The first sentence states the core purpose, and the second adds usage guidance. Every word earns its place; there is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (0 params, no output schema), the description is reasonably complete. It covers the main return content ('list and descriptions') and the recommended usage context. However, it does not detail the exact structure of the returned data (e.g., field names or types), which would be expected without an output schema. Still, for a simple listing tool, it is sufficient for initial selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the baseline is 4 according to the rubric. The description adds value by explaining the returned content (list and descriptions) and its use, even though it does not need to provide parameter-level semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'returns the list and descriptions of registered projects.' The verb '반환' (returns) and the resource '프로젝트 목록과 설명' (list and descriptions of projects) make the purpose unambiguous. It also distinguishes itself from siblings like add_project/remove_project (mutations) and search_code/read_file (content retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context for when to use: 'call before searching to understand which projects exist and narrow down the search scope based on the descriptions.' This provides clear usage guidance but does not explicitly mention when not to use or alternative tools, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses a key constraint: only files within registered project paths can be read, which is valuable. It does not mention error handling or return format, but for a read-only operation this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences: first states the action, second gives usage, third gives a limitation. No redundant information, front-loaded with the primary purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with full schema coverage and no output schema, this description covers purpose, usage, and constraints sufficiently. The sibling tools have different functions, so no further context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes 'path' as an absolute path, achieving 100% coverage. The description adds no extra parameter details beyond referencing a 'specific file', so it meets the baseline but doesn't enhance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'reads and returns' with the resource 'specific file', clearly distinguishing it from sibling tools like search_code and get_file_tree. It also adds context about checking full contents of files from search results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use: to check the full contents of a file found in search results. It also mentions the constraint of only registered project paths, implying not for external files, but doesn't name specific alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.