Server Quality Checklist

Tool Scores

• add_issueD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Tool has no description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tool has no description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has no description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tool has no description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Tool has no description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• process_issuesC
  2.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It does not disclose whether processing is destructive, requires permissions, or any side effects. The only hint is 'summary of the issues', suggesting it may be a read operation, but 'process' implies mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (two sentences) but lacks substance. While concise, it is under-specified and does not effectively communicate the tool's purpose or behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and a vague description, the tool is incomplete for an agent. The output schema exists but is not shown; the description only mentions a summary. More context about what 'process' does is needed, especially without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100% (vacuously). Baseline 3 applies. The description adds no param info because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says 'process the issues' but does not specify what processing entails. It is vague and does not distinguish from sibling tools like add_issue, ai_try_solve, and assign_expert.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ai_try_solveC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It discloses the return value (solution or fallback message) but does not mention side effects (e.g., does it modify data?), required permissions, or what happens on failure beyond the message. The behavioral impact is unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence plus @param lines. The @param lines add little value and could be removed. It is not excessively long, but the Javadoc-style formatting is slightly verbose for the content provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and an output schema present, the description covers the return value but lacks detail on parameter formats or edge cases. With 4 required params, more guidance would improve completeness. The fallback message provides useful context for when the tool fails.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. The @param comments only repeat parameter names (e.g., 'The description of the issue') without adding meaning, allowed values, or format. With no enums or further details, the semantics are weak.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose: 'try to solve the issue with AI first'. It clearly identifies the verb-resource pair (try solve issue) and distinguishes from siblings like assign_expert which assigns experts. However, it does not explicitly differentiate from add_issue or process_issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a workflow by stating 'try to solve with AI first' and the return message suggests assigning an expert if no solution is found. However, it lacks explicit guidance on when to use this tool versus alternatives, such as prerequisites or cases where it should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assign_expertC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It describes analysis and selection but does not disclose whether the tool performs an actual assignment (side effect) or is read-only. No information about required permissions, data persistence, or side effects is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is straightforward and free of unnecessary words. It could benefit from a more structured format (e.g., bullet points) but is not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality but lacks usage context and behavioral details. Given an output schema exists, return values are likely documented elsewhere, but the absence of when-to-use guidance and side-effect disclosure leaves gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'description' has 0% schema description, so the tool description's mention of 'problem description' clarifies its purpose. However, no additional constraints like expected length, format, or examples are given, meaning the description adds minimal value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes a problem description, classifies it into categories, and chooses an expert. This gives a specific verb-resource pairing. However, it does not explicitly differentiate from sibling tools like 'add_issue' or 'ai_try_solve', missing context on how assignment differs from issue creation or AI solving.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites, scenarios where this tool is appropriate, or cases where another sibling tool should be used instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.