Server Quality Checklist

Tool Scores

• extract_keywordsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions morphological analysis for keyword extraction but doesn't describe traits like language support, performance characteristics, error handling, or output format. For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Japanese that directly states the tool's function. It's front-loaded with the core action ('形態素解析でキーワードを抽出'), but could be slightly more structured by including key details. There's no wasted text, making it concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., list of keywords with scores), how errors are handled, or any dependencies. For a tool with 2 parameters and complex text processing, more context is needed to ensure proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for 'text' and 'top_n'. The description doesn't add any meaning beyond the schema, such as explaining how 'top_n' interacts with the extraction algorithm or providing examples. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool performs keyword extraction using morphological analysis, which is a clear purpose. However, it doesn't specify what type of keywords (e.g., nouns, named entities) or how it distinguishes from sibling tools like 'analyze_survey' or 'generate_wordcloud' that might also process text. The description is somewhat vague about the exact nature of the extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention any context, prerequisites, or exclusions, such as when to prefer 'extract_keywords' over 'analyze_survey' or 'generate_wordcloud'. It lacks explicit usage instructions, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_surveyC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the tool generates HTML reports but doesn't cover critical aspects like whether it modifies input files, requires specific permissions, has rate limits, or handles errors. For a tool with 6 parameters and no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Japanese that front-loads the core functionality. It wastes no words and directly states what the tool does, making it highly concise and well-structured for its purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no output schema, no annotations), the description is incomplete. It lacks information on behavioral traits, output details, error handling, and usage guidelines. While concise, it doesn't provide enough context for an agent to fully understand how to invoke and interpret results from this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by listing report components (morphological analysis, WordCloud, graphs), which loosely relates to parameters like 'analysis_axes' and 'enable_ai_analysis'. However, it doesn't provide additional syntax, format details, or usage context for the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: analyzing survey CSV files and generating HTML reports with specific components (morphological analysis, WordCloud, graphs). It uses specific verbs ('analyze', 'generate') and identifies the resource ('survey CSV files'). However, it doesn't explicitly differentiate from sibling tools like 'extract_keywords' or 'generate_wordcloud', which might offer overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'extract_keywords' or 'generate_wordcloud', nor does it specify prerequisites, ideal scenarios, or exclusions. Usage is implied through the description but lacks explicit context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_wordcloudC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool generates a WordCloud but doesn't explain what that entails (e.g., image creation, file output, processing steps). It misses details like whether it's read-only, destructive, or has side effects (e.g., saving files), which is critical for a tool with output operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Japanese that directly states the tool's function. It's front-loaded with the core purpose and has no unnecessary words, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a tool that generates visual output and has no output schema or annotations, the description is incomplete. It doesn't cover behavioral aspects (e.g., file creation, error handling) or provide context on when to use it, leaving gaps in understanding the tool's full operation and integration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are documented in the schema itself. The description adds no additional meaning beyond the schema, such as explaining how parameters interact or providing examples. With high schema coverage, the baseline score is 3, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: generating a WordCloud from text data. It specifies the verb ('generate') and resource ('WordCloud'), making it understandable. However, it doesn't distinguish this tool from its siblings (analyze_survey, extract_keywords, update_ai_analysis), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, exclusions, or comparisons with sibling tools like extract_keywords, which might also process text. This lack of context leaves the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_ai_analysisA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's function (updating HTML reports with analysis) and prerequisites, but doesn't mention potential side effects (e.g., overwriting existing content), error conditions, or output format. The description adds value with the usage checklist and subscription context, but lacks details on mutation behavior, permissions, or response expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a purpose statement, a numbered checklist, and a usage condition. However, it's somewhat verbose for a tool description—the checklist could be more concise or moved to documentation. The information is front-loaded with the main purpose, but the detailed prerequisites might be better suited for a separate usage guide.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (updating reports with structured data), no annotations, and no output schema, the description does a reasonably complete job. It explains the tool's purpose, provides detailed prerequisites, and specifies usage context. However, it lacks information on what the updated report looks like, error handling, or confirmation of changes, which would be helpful for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions '課題と解決策' (issues and solutions) which aligns with the 'issues' and 'solutions' parameters, but provides no additional syntax, format, or usage details. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '既存のHTMLレポートにClaude Codeが分析した課題と解決策を追加します' (adds issues and solutions analyzed by Claude Code to an existing HTML report). This specifies the verb (add/update), resource (HTML report), and content (issues and solutions). However, it doesn't explicitly differentiate from sibling tools like 'analyze_survey' or 'extract_keywords', which appear to be separate analysis tools rather than report updaters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: '使用前に必ず以下を実施してください' (before use, always do the following) with a 5-step checklist for data preparation. It also specifies when to use this tool: 'APIキーなしでClaude Codeサブスクリプションのみで利用する場合に使用します' (use when using only Claude Code subscription without API key). This gives clear context and prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.