Server Quality Checklist

Tool Scores

• fix_codeB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. The description does not disclose behavioral traits such as whether the tool modifies files, requires authentication, or has rate limits. It only lists inputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and no output schema, the description omits return value, behavior for different severities, and how to choose between siblings. Incomplete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description repeats parameter names without adding additional semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Generate a fix' and resource 'specific code issue'. It distinguishes from siblings like review_code and ask_codepeel by focusing on fixing rather than reviewing or asking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when there is a code issue to fix, but does not provide when-not-to-use or alternatives. No explicit comparison with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_codepeelA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool can analyze and answer questions, which implies a read-only operation, but does not explicitly confirm non-destructiveness, rate limits, or authentication requirements. This is acceptable for a simple query tool but leaves some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the primary purpose and then listing capabilities. It is efficient with no redundant wording, though it could be slightly more structured (e.g., using bullet points for capabilities). Overall, it earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description adequately covers its purpose and optional input. It does not specify return values, but for a conversational tool this is acceptable. It is complete enough for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have descriptions in the schema (100% coverage). The description adds value by emphasizing that the diff parameter is optional and for context, reinforcing its purpose beyond the schema. This extra context helps the agent understand how to use the diff effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Ask CodePeel a question about code.' It lists specific capabilities like analyzing patterns, explaining issues, suggesting improvements, and answering architecture questions, which makes the tool's function distinct from siblings like fix_code (fixing) and review_code (reviewing). This provides a comprehensive and unambiguous definition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for code-related questions but does not explicitly state when to use this tool over siblings like fix_code or review_code. It mentions an optional diff for context but lacks guidance on prerequisites or exclusion criteria, resulting in a moderately clear but incomplete directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_creditsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description implies a read-only operation via the word 'Check'. It doesn't disclose any side effects or required permissions, but for a zero-parameter tool, the behavioral disclosure is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It efficiently communicates the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description adequately covers the tool's action and scope. It could optionally mention that it returns the information, but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100%, so baseline is 4. The description adds value by specifying what data is checked (balance, plan tier, usage), which the schema does not convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Check' and resource 'your CodePeel account balance, plan tier, and usage', clearly distinguishing it from siblings like ask_codepeel, fix_code, and review_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives, nor does it mention when not to use it. Given the presence of siblings, this is a gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_codeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully explains behavior: it analyzes for bugs, security issues, and violations, and returns findings with severity, explanation, and fixes. It does not mention side effects or permissions, but these are not needed for a read-only analysis. Transparent enough for an AI agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose and returns, second states input format. No unnecessary words. Front-loaded with key action and result.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only 2 parameters (1 required) and no output schema. The description explains what it returns (findings with severity, explanation, fixes). This is sufficient for an agent to use it correctly without further detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters with descriptions that match the tool description. The description adds no new information beyond what the schema provides (e.g., 'diff' is already described as unified diff). Schema description coverage is 100%, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool reviews code changes for bugs, security issues, and best practice violations. It uses a specific verb ('review') and identifies the resource ('code changes'). It distinguishes itself from sibling tools like 'fix_code' (which fixes) and 'ask_codepeel' (likely for questions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on input format: 'Pass a unified diff or file contents.' It implies this is for reviewing changes before merging, but does not explicitly exclude other scenarios or mention when to use alternatives. Clear context with room for more precision.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.