Server Quality Checklist

Tool Scores

• update_productC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers none. '更新する' (update) implies a mutation operation, but the description doesn't mention permission requirements, whether updates are partial or complete, side effects, error conditions, or response format. For a mutation tool with 10 parameters and no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three Japanese characters. While this represents under-specification rather than ideal conciseness, according to the scoring framework, it's not penalized in this dimension. The description wastes no words and is front-loaded, earning full points for conciseness despite its inadequacy in other areas.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, mutation operation), lack of annotations, and absence of an output schema, the description is severely incomplete. It doesn't explain what the tool returns, error conditions, or behavioral constraints. For a product update tool in what appears to be an e-commerce context, this minimal description leaves critical gaps for an agent to understand how to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly with Japanese descriptions. The tool description adds no additional parameter context beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no parameter information in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '商品を更新する' (Update product) is a tautology that essentially restates the tool name 'update_product' in Japanese. It doesn't specify what aspects of a product can be updated or differentiate this tool from its sibling 'create_product' beyond the basic verb. The purpose is clear at a high level but lacks specificity about scope or capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'create_product' or 'get_product'. There's no mention of prerequisites (e.g., needing an existing product ID), constraints, or typical use cases. An agent would have to infer usage from the parameter schema alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a read operation ('取得する'), which implies it's non-destructive, but doesn't mention other important traits like authentication requirements, rate limits, error conditions, or what '詳細情報' (detailed information) specifically includes. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence in Japanese that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple read operation, though it could be slightly more informative without losing conciseness. There's no structural issue like burying key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, 100% schema coverage) and lack of annotations or output schema, the description is incomplete. It doesn't explain what '詳細情報' (detailed information) includes in the response, potential error cases (e.g., invalid product_id), or how this differs from sibling tools. For a read operation with no output schema, more context about return values would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'product_id' clearly documented as '商品ID' (product ID) in the schema. The description doesn't add any meaning beyond this—it doesn't explain what format the ID should be in, where to find it, or provide examples. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '商品の詳細情報を取得する' (Get detailed product information) clearly states the verb '取得する' (get) and resource '商品の詳細情報' (detailed product information), making the purpose understandable. However, it doesn't distinguish this tool from its sibling 'list_products' (which presumably lists multiple products rather than getting details of a specific one), so it doesn't reach the highest clarity level.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_products' for browsing products or 'create_product' for adding new ones. It doesn't mention prerequisites (e.g., needing a valid product_id) or contextual constraints, leaving the agent to infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_productC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While '削除する' (delete) implies a destructive mutation, the description doesn't specify whether deletion is permanent, requires specific permissions, has side effects (e.g., cascading deletions), or provides confirmation feedback. This is inadequate for a destructive tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('商品を削除する') that directly states the tool's action without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical context: what happens on success/failure, whether deletion is reversible, permission requirements, or error conditions. Given the complexity and risk of deletion operations, this minimal description doesn't provide enough information for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (the product_id parameter is fully documented in the schema as '商品ID'), so the baseline is 3. The description adds no additional parameter semantics beyond what the schema already provides—it doesn't explain format constraints, validation rules, or example values for product_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '商品を削除する' (deletes a product) clearly states the verb (delete) and resource (product), making the basic purpose understandable. However, it doesn't differentiate this destructive operation from its sibling tools (create_product, get_product, list_products, update_product) beyond the obvious verb difference, missing opportunities to clarify scope or constraints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., product must exist), consequences (e.g., irreversible deletion), or relationships with sibling tools (e.g., use get_product first to verify). This leaves the agent without contextual usage instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_productsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('取得する' - get) but doesn't describe what '一覧' (list) entails (e.g., pagination, sorting, filtering beyond the limit parameter), return format, or any constraints like rate limits or authentication needs. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient phrase ('商品一覧を取得する') with zero wasted words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a list of product objects, pagination details) or behavioral aspects like error handling. For a list-retrieval tool with no structured context, this leaves the agent under-informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'limit' parameter fully documented (default: 50, min: 1, max: 250). The description adds no parameter information beyond what the schema provides, so it meets the baseline of 3 for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '商品一覧を取得する' (Get product list) states a clear verb ('取得する' - get) and resource ('商品一覧' - product list), which establishes the basic purpose. However, it doesn't distinguish this from its sibling 'get_product' (which presumably gets a single product), leaving ambiguity about when to use each tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_product' (for single products) or other siblings. There's no mention of prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_productC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. 'Create' implies a write/mutation operation, but the description doesn't mention required permissions, whether creation is idempotent, what happens on duplicate titles, or what the response contains. For a mutation tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient Japanese sentence that states the core purpose without any wasted words. It's appropriately sized for a creation tool and front-loads the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex product creation tool with 9 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what constitutes a successful creation, what data is returned, or how to handle the various nested structures (images, options, variants). The agent would need to guess about the mutation's behavior and results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with all 9 parameters well-documented in Japanese. The description adds no parameter information beyond what's already in the schema. According to guidelines, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description '新しい商品を作成する' (Create a new product) clearly states the verb ('create') and resource ('product'), making the purpose immediately understandable. It doesn't explicitly differentiate from siblings like 'update_product' or 'delete_product', but the verb 'create' inherently distinguishes it as a creation operation rather than modification or deletion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'update_product' or 'list_products'. There's no mention of prerequisites, constraints, or appropriate contexts for creation versus other product operations. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.