Server Quality Checklist

Tool Scores

• review_codeB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It clearly states that the tool does not call an LLM directly (it '不直接调用 LLM'), which is a key behavioral trait. However, it does not disclose what the output looks like, whether it requires specific permissions, or if it has side effects (e.g., saving prompts). For a tool that builds prompts, return format and any side effects are important for agent transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently communicates the core purpose and a critical behavioral constraint. It is front-loaded and contains no extraneous information. It earns its place by being both concise and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool builds a prompt for code review, the description is fairly complete: it says what it does (generate a prompt) and what it does NOT do (call an LLM). However, with no output schema and no behavioral transparency about return values or side effects, the agent may not know what to expect from the output. The sibling tools suggest 'parse_review_score' consumes this output, but the description doesn't confirm that. A slightly longer description explaining output format would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond the schema: parameter names and their descriptions in the schema already cover the three parameters (code, style, commitMessage). The description does not clarify how 'code' should be formatted, how 'style' affects the generated prompt, or what 'commitMessage' does. It adds marginal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('构建','generate') and resource ('LLM 提示词','LLM prompt') for code review and scoring. It also distinguishes itself by noting it does NOT call an LLM directly, which differentiates it from the sibling tools 'review_diff' and 'review_file' that likely perform actual code review. However, the description is in Chinese, which may limit understanding for non-Chinese-speaking agents, but it still conveys purpose effectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for building a prompt for code review, but it lacks explicit guidance on when to use this tool vs alternatives like 'review_diff' or 'review_file'. It does not state when NOT to use it, nor provide context about prerequisites or integration with other tools like 'parse_review_score'. The sibling tools suggest a workflow, but the description doesn't clarify the role of this tool within it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_fileB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool builds a prompt and does not call the LLM directly, which is a critical behavioral trait. However, it does not disclose what the output format is (e.g., returned string, saved file), nor any side effects or prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is compact and front-loaded with the key action and resource. No extraneous words or repetition. Every part of the sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is incomplete for a tool with 4 parameters and no output schema. It does not explain what the tool returns (e.g., the prompt as a string, a file path, or something else). The missing return value is a significant gap that hinders an agent from correctly using the tool's output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter already has a description. The overall description adds context that the parameters are used to construct a review prompt, which enhances understanding. However, it does not add specific meaning beyond what the schema provides, such as how 'style' influences the prompt or the role of 'commitMessage'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds an LLM prompt for single file review and scoring, and explicitly notes it does not directly call the LLM. This verb+resource pairing is specific. However, it does not explicitly differentiate from siblings like 'review_code' or 'review_diff', relying on the indirect implication that those are direct reviewers rather than prompt builders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It implies the tool is for building a prompt rather than performing a direct review, but does not state conditions, exclusions, or mention sibling tools. The agent is left to infer usage context without clear direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_review_scoreA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. The description indicates the tool extracts a score based on a specific pattern, but it does not disclose what happens if the pattern is not found (e.g., returns null, throws error, or returns empty string). It also does not mention any side effects (e.g., whether the review text is modified or if the tool requires specific permissions). The description is adequate but leaves ambiguity about failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and to the point, using a single sentence to convey the core functionality. It front-loads the key information about what the tool does and the expected input format. While it is concise, it could be improved by adding a brief note on failure behavior without significantly increasing length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has only one simple parameter and no output schema, the description is reasonably complete for its simplicity. However, it lacks details about return format (e.g., whether it returns a number or string) and error handling. Since there is no output schema, a brief mention of what the output looks like would enhance completeness. The description is adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% coverage with one parameter 'reviewText', and the description adds context by specifying that the text should contain '总分:XX分' format. However, it does not elaborate on the syntax of the expected format beyond the example, such as whether the score must be an integer or can be decimal, or if there are constraints on the value range. Given the schema already describes the parameter well, the description adds marginal value but not enough to rise above baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '从审查文本中解析评分（提取 '总分:XX分' 格式）', which translates to 'parse review scores from review text (extract format '总分:XX分')'. It specifies the exact verb (parse), the resource (review text), and the specific format to extract, distinguishing it from sibling tools like review_code, review_diff, and review_file, which may involve different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description does not explicitly mention when not to use this tool or alternatives, it implies usage when a review text contains a score in the specified format. The context signals and sibling tool names suggest this is a specialized parsing tool, so an agent would infer to use it only when needing to extract a score, not for other review-related tasks. However, it lacks explicit guidance on what to do if the format is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_diffA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It clearly discloses that the tool does not invoke an LLM, which is critical behavioral context. It does not describe return format or side effects, but for a prompt-building tool, the non-LLM-calling behavior is the key transparency point.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core purpose and key behavioral constraint (no direct LLM call). There is no superfluous content; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with no output schema, the description is complete enough. It explains the tool's function and boundary (no LLM call). The missing output spec is not critical since the tool's purpose is to produce a prompt, and the user likely knows what a prompt is.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add any parameter-specific context beyond what the schema already provides (e.g., no examples, no explanation of how 'style' affects output). The description remains high-level about the tool's function.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to build an LLM prompt for reviewing and scoring Git diff changes, explicitly noting that it does not directly call an LLM. It distinguishes itself from siblings by focusing on prompt construction for diff review, not general code review (review_code) or file review (review_file).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when an LLM prompt for diff review is needed, and the sibling names suggest alternatives for code or file review. However, it lacks explicit guidance on when not to use this tool versus the alternatives, and there is no mention of prerequisites like having diff content ready.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.