Server Quality Checklist

Tool Scores

• check_projectA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it traverses files and runs rules, implying a potentially expensive operation. However, it does not mention performance impact, permissions needed, error handling (e.g., invalid directory), or side effects (none expected).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bullet points and clear sections. It is not overly verbose, though the initial sentence could be more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers purpose, parameter, and output summary well. It lacks details on failure modes or exact return structure, but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'directory' is explained in the description (path to scan, defaults to current directory), adding meaning beyond the schema's type and default. Schema coverage is 0% but the description compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scans a project directory for code quality, listing specific metrics like file counts, language distribution, error counts, top 10 files, and overall score. It distinguishes from sibling tools like analyze_file (single file) and list_rules (list rules).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives (e.g., analyze_file for single files, review_diff for diffs). The description only explains what it does, not the decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_fileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It details the specific rule categories (SEC001-012, COMPLEX001-005, STYLE001-004), giving transparent insight into what checks are performed. It does not disclose side effects but the tool is read-only by nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with an overview, rule categories list, and an Args section. While not extremely concise, every sentence adds information. It is well-organized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite detailed rule listings, the description lacks information about return values or error handling (e.g., file not found). With no output schema, the agent must guess the response format. Some behavioral gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description compensates by explaining file_path meaning: relative to project root or absolute path. This adds significant value beyond the schema, which only provides type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it analyzes code quality, complexity, and security for a single file. It lists specific rule categories, and the sibling tools (check_project, list_rules, review_diff) provide context that this is file-level analysis, distinguishing it from project-level or diff-focused tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for single-file analysis but does not explicitly state when to use it versus alternatives like check_project or review_diff. No when-not-to-use guidance or prerequisites are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_diffA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It details the specific checks performed (hardcoded keys, debug logs, TODO/FIXME/HACK, long lines) and mentions it reviews both staged and unstaged changes. However, it does not clarify if the tool is read-only or what happens if no changes are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a two-line summary followed by a bullet list of checks. Every sentence is meaningful, no fluff, and the structure is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers the tool's purpose and checks well. It lacks details on the return format or how results are presented, but the context is largely complete for a stateless review tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4. The description adds value by explaining the purpose and checks performed, which compensates for the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reviews uncommitted git changes and lists specific checks (hardcoded keys, debug statements, TODO/FIXME/HACK markers, long lines). It distinguishes itself from siblings like analyze_file (which focuses on individual files) and check_project (broader scope) by targeting the diff of staged and unstaged changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage before committing but does not explicitly state when to use this tool over alternatives like analyze_file or check_project. No exclusion criteria or recommended contexts are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rulesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly indicates this is a read-only listing operation and describes the return fields (ID, name, description, category, default severity). However, it does not disclose potential ordering, filtering, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences with the purpose front-loaded. Every word adds value, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description fully explains what the tool does and what it returns. It is complete for a simple listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema description coverage is 100% (vacuous). The description adds context about what the tool returns, but since there are no parameters, it is not deficient. The baseline of 3 is exceeded because the description still provides helpful output information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all registered code review rules, using the verb '列出' (list) and resource '代码审查规则' (code review rules). It distinguishes from sibling tools (analyze_file, check_project, review_diff) which involve analysis or checking rather than listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for AI selective calling but does not explicitly state when to use this tool versus alternatives or provide exclusions. Usage is implied as a prerequisite for selecting rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.