Server Quality Checklist

Tool Scores

• agentic_issue_analyzeC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the tool performs AI analysis and automatic label assignment, it doesn't describe what happens during execution - whether it makes changes to the issue, what permissions are required, whether it's a read-only analysis tool or modifies the issue, what happens if analysis fails, or what the typical response looks like. For a tool that appears to modify issues (label assignment implies mutation), this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief (one phrase) and mixes Japanese and English terms ('IssueAgent実行', 'Label自動付与'), which may hinder clarity for English-only agents. While concise, it's arguably too brief - it doesn't provide enough context for the tool's purpose and behavior. The information is front-loaded but insufficiently detailed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns, what side effects it has (particularly important given it mentions 'automatic label assignment'), or provide enough context about the AI analysis process. The description leaves too many unanswered questions about how the tool behaves and what results to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, with all three parameters clearly documented in the schema itself. The description adds no additional parameter information beyond what's already in the schema - it doesn't explain how the parameters interact, provide examples of valid inputs, or clarify any constraints not captured in the schema. With complete schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool performs 'AI analysis' and 'automatic label assignment' on issue content, which provides a general purpose. However, it's somewhat vague about the specific nature of the analysis (e.g., what type of AI analysis, what labels are assigned) and doesn't clearly distinguish this tool from potential siblings like 'agentic_review_execute' which might also involve issue analysis. The description mixes Japanese and English terms, which could cause confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or specific scenarios where this tool is appropriate compared to sibling tools like 'agentic_review_execute' or 'agentic_pr_create'. There's no indication of when NOT to use this tool or what alternatives exist for similar functionality.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_codegen_executeC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions AI-driven code generation and test automation but doesn't specify what happens during execution (e.g., does it modify files, run tests, require specific permissions, or have rate limits?). For a tool that likely performs significant operations, this lack of detail is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with a single phrase, which is efficient and front-loaded. However, it's arguably too brief for a tool with significant functionality, potentially under-specifying rather than being optimally concise. It could benefit from slightly more detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity implied by AI-driven code generation and test automation, no annotations, and no output schema, the description is incomplete. It doesn't cover what the tool returns, error conditions, or behavioral aspects like side effects. For a tool with 4 parameters and likely substantial operations, this minimal description is inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters (description, issue_number, priority, title). The description doesn't add any meaning beyond this, such as explaining how parameters interact (e.g., how issue_number relates to GitHub) or providing examples. Baseline 3 is appropriate when the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'CodeGenAgent実行 - AI駆動コード生成・テスト自動生成' states the tool executes AI-driven code generation and test automation, which is a clear purpose. However, it's somewhat vague about the specific verb ('実行' means execute/run) and doesn't explicitly distinguish this code generation tool from sibling tools like 'agentic_pr_create' or 'agentic_review_execute' that might also involve code-related operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing a GitHub issue), context for code generation tasks, or how it differs from sibling tools such as 'agentic_issue_analyze' or 'agentic_pr_create'. Users must infer usage from parameter names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_coordinator_decomposeC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'task decomposition (DAG construction) and agent selection', which suggests a planning/coordination operation, but lacks details on side effects (e.g., whether it modifies data, requires specific permissions, or has rate limits). For a tool with no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise (one short phrase) and front-loaded with the core purpose. However, it could be more structured by separating concepts (e.g., 'Decomposes tasks into a DAG and selects agents for execution by CoordinatorAgent'). The mixed language (Japanese/English) slightly reduces clarity but doesn't add unnecessary length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity implied by 'DAG construction' and 'agent selection', along with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a plan, agent assignments), how errors are handled, or dependencies on other tools. This leaves the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters (description, issue_number, title) with clear descriptions. The tool description adds no additional parameter semantics beyond what's in the schema, such as format examples or constraints. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool performs 'task decomposition (DAG construction) and agent selection' for a 'CoordinatorAgent', which provides a general purpose. However, it's somewhat vague about what specific resources are involved (e.g., GitHub issues, code tasks) and doesn't clearly distinguish from siblings like 'agentic_issue_analyze' or 'agentic_codegen_execute'. The Japanese/English mix adds minor ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies it's for task decomposition, but it doesn't specify prerequisites, timing, or exclusions relative to siblings like 'agentic_issue_analyze' (which might analyze issues) or 'agentic_codegen_execute' (which might execute code). Usage is only loosely implied by the tool's name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_kpi_collectC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions collecting KPIs and generating dashboards, but fails to describe critical behaviors such as whether this is a read-only or mutating operation, what permissions are required, how data is sourced, or any rate limits. This leaves significant gaps for a tool that likely involves data processing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with just two phrases ('KPI収集・ダッシュボード生成'), which is efficient and front-loaded. However, it is under-specified rather than optimally concise, as it lacks necessary details for full understanding, slightly reducing its effectiveness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity implied by KPI collection and dashboard generation, with no annotations and no output schema, the description is incomplete. It doesn't explain what KPIs are collected, how the dashboard is generated or returned, or any behavioral traits, making it inadequate for the agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'period' parameter well-documented as an enum for aggregation periods. The description does not add any meaning beyond what the schema provides, as it doesn't explain how the period affects KPI collection or dashboard generation. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'KPI収集・ダッシュボード生成' states the tool collects KPIs and generates dashboards, which gives a general purpose. However, it's vague about what specific resources or data sources are involved, and it doesn't distinguish this from sibling tools like 'agentic_metrics_view' or 'agentic_issue_analyze' that might also handle metrics or analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, context for KPI collection, or comparisons to sibling tools such as 'agentic_metrics_view' for viewing metrics or 'agentic_issue_analyze' for analysis, leaving the agent with no usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_pr_createC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It mentions AI generation for descriptions, which adds some context, but fails to disclose critical traits: whether this is a read-only or mutative operation (implied creation suggests mutation), authentication needs, rate limits, or what happens on execution (e.g., PR submission to GitHub). For a tool with no annotations, this leaves significant gaps in understanding behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two parts separated by a dash, front-loading the main action ('PRAgent実行') and specifying outcomes. It wastes no words, though it could be slightly clearer in English translation. Every phrase contributes to the purpose, making it efficient despite language barriers.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (PR creation with AI features), no annotations, no output schema, and 2 parameters, the description is incomplete. It doesn't explain return values, error handling, or integration details (e.g., with GitHub). For a tool that likely involves mutative actions and AI processing, more context is needed to ensure the agent can use it effectively without structured support.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear parameter descriptions in Japanese ('ブランチ名（オプション）' for branch_name, 'GitHub Issue番号' for issue_number). The description adds no parameter semantics beyond the schema, such as format examples or constraints. With high schema coverage, the baseline is 3, as the schema adequately documents parameters without extra help from the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'PRAgent実行 - PR自動作成・説明文AI生成' states the tool creates PRs and generates AI descriptions, which gives a general purpose. However, it's somewhat vague in Japanese and doesn't clearly distinguish this PR creation tool from sibling tools like 'agentic_codegen_execute' or 'agentic_review_execute' that might also involve PR-related operations. It specifies the verb (create/generate) and resource (PR/description) but lacks precision about scope or differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context (e.g., after issue analysis), or exclusions. Given sibling tools like 'agentic_issue_analyze' that might precede PR creation, the lack of usage guidelines leaves the agent to infer timing, which is insufficient for clear tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_metrics_viewC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. The description only states what the tool does (display a dashboard) without mentioning any behavioral traits like whether it's read-only, requires authentication, has rate limits, or what format the display takes. For a tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise phrase that directly states the tool's function. It's appropriately sized for a simple tool with no parameters, though it could be slightly more informative. There's no wasted verbiage, and it's front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and 0 parameters, the description is incomplete. It doesn't explain what the dashboard displays (e.g., metrics format, time range, data source), how the display is rendered (e.g., visual vs. textual), or any behavioral aspects. For a dashboard tool, users need more context about what they'll see and how it works.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema). The description doesn't need to explain parameters since none exist. This meets the baseline of 4 for tools with no parameters, as there's nothing to compensate for and no schema details to elaborate upon.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '識学理論KPIダッシュボード表示' translates to 'Shikigaku Theory KPI Dashboard Display' which indicates showing a dashboard with KPI metrics related to Shikigaku theory. This provides a general purpose (displaying a dashboard) but lacks specificity about what resources or data are involved and doesn't differentiate from sibling tools like 'agentic_kpi_collect' which might collect similar metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of context, prerequisites, or comparisons to sibling tools like 'agentic_kpi_collect' (which might collect metrics) or 'agentic_issue_analyze' (which might analyze related issues). The user must infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_review_executeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions three types of activities (static analysis, security scanning, quality assessment), it doesn't describe what happens during execution: whether it's read-only or modifies code, what permissions are needed, what the output looks like, or any rate limits. For a tool with 'execute' in its name and no annotations, this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single phrase listing three activities. While efficient, it might be too terse for a tool called 'execute' that presumably performs significant operations. Every word earns its place, but more context could be beneficial given the tool's apparent complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool name suggests execution/action, no annotations, no output schema, and sibling tools that might overlap, the description is incomplete. It doesn't explain what 'execution' entails, what results to expect, or how this differs from related tools. For a potentially complex execution tool in a suite of agentic tools, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no additional parameter information beyond what's in the schema - it doesn't explain how 'issue_number' relates to the review execution or what happens with 'target_files'. The baseline score of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs review execution with three specific activities: static analysis, security scanning, and quality assessment. It uses the Japanese term '実行' (execute) which aligns with the tool name. However, it doesn't explicitly differentiate from sibling tools like 'agentic_issue_analyze' or 'agentic_pr_create' which might have overlapping review-related functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, appropriate contexts, or comparisons to sibling tools like 'agentic_issue_analyze' (which might analyze issues) or 'agentic_pr_create' (which might create pull requests). The agent must infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.