Server Quality Checklist

Tool Scores

• update_content_nodeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully handles behavioral disclosure. It states the tool modifies nodes, requires graph input, outputs graph+changed ids+diff summary, and important constraint: nodes with user_verified=true reject changes. Does not cover idempotency or side effects, but sufficient for safe use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with clear sections (when to use, input, output, note). Each sentence earns its place—no fluff. Front-loaded with usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains return format (graph, changed_node_ids, diff_summary). It provides behavioral note about validation. For a complex tool with nested object input and 5 params, this is adequately complete. Could mention error handling or edge cases, but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds little beyond schema: it lists parameters and notes optionality (new_type optional, mark_verified default true). But the schema already describes them. No significant enrichment of parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool updates a node's content with specific examples (mail text, time, participants). It distinguishes itself from siblings implicitly by focusing on modification, while extract_content_graph extracts and analyze_impact likely analyzes. However, no explicit distinction from alternatives is given.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'when to use' section (ユーザーがノードの内容を変更するとき) and guidance to call after extract_content_graph. Includes a constraint ('user_verified=true rejects changes') but does not specify when NOT to use or compare with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_impactA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully covers behavioral context. It describes the output fields including risk_level and warning, and a critical note about empty affected_node_ids meaning no editing needed. It does not mention auth or side effects, but as an analysis tool, it likely has no destructive behavior. The description adds significant value beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly structured with clear sections for usage, inputs, outputs, and notes. It is concise (5 lines) with no redundant information, making it easy for an AI agent to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity with nested input objects and no output schema, the description provides complete context: when to use, exact input expectations, full output structure, and a critical behavioral note. No gaps remain for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds meaning by labeling graph as '更新済みContentGraph' (updated ContentGraph) and describing the output structure that clarifies what the parameters lead to. This goes beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear usage context: 'update_content_nodeで変更したノードの影響範囲を確定するとき', which precisely states that this tool is used after update_content_node to determine the impact range. This distinguishes it from siblings extract_content_graph and update_content_node, as it focuses on impact analysis after changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use: after update_content_node. It also includes a note about when editing is unnecessary (if affected_node_ids is empty), providing practical guidance. However, it does not explicitly mention when not to use or alternative tools, though siblings are clearly different in purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_content_graphA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses high cost, character limit, output structure (nodes/edges with fields), and the need to pass the graph to subsequent tools. Lacks details on side effects or error conditions, but otherwise thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Structured with clear labels (【いつ使う】, 【入力】, 【出力】, 【注意】) and front-loaded with usage guidelines. Every sentence is necessary and concise, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description describes the output structure (ContentGraph with nodes/edges and fields like group_id) thoroughly. It also provides usage context for subsequent steps, making it complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter (text) has 100% schema coverage, but the description adds specific examples of acceptable content and a maximum character length (50,000), adding meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the purpose: to graphify text from internal emails, meeting requests, agendas, etc., for the first time only. It explicitly distinguishes from sibling tools (analyze_impact, update_content_node) by focusing on initial extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (first time only) and when not to (second time or later for same document group), with reasoning about high cost. It implies alternatives: subsequent tools for later steps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.