Server Quality Checklist

Tool Scores

• analyzeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description does not contradict that. However, the description adds no behavioral context beyond the annotation, such as required permissions, rate limits, or side effects. For a read-only analytics tool, this is acceptable but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short (two sentences) and front-loaded with a generic lead followed by specifics. It is efficient but the first sentence 'Analytics and reporting' is vague. Could be slightly improved by leading with the specific actions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 19 parameters and 4 distinct modes with no output schema, the description is too high-level. It does not explain the modes or common use cases, leaving the agent to rely solely on the schema. For a complex tool, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema documentation covers 100% of parameters with detailed descriptions. The tool description adds no extra parameter information. With full schema coverage, baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states four specific analytical functions (token savings, cost audit, AEO reports, articles), which is more specific than a generic verb. However, it does not differentiate from sibling tools like 'report', leaving some ambiguity about when to use this tool versus 'report'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings (inspect, lookup, report, search_services). The description only lists capabilities without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspectB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate non-read-only and non-idempotent behavior. The description lists write modes (submit, propose, review, snapshot) but does not disclose side effects, required permissions, or rate limits. Minimal additional value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose, lists all 8 modes concisely, and ends with auto-detection note. It is efficient given the complexity of 35 parameters, though a bullet list could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers modes and parameter usage adequately due to rich schema, but lacks details on return values, error handling, and prerequisites. No output schema means agents must infer response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds high-level mode context and auto-detection hints, but does not deepen understanding of individual parameters beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for colony health inspection with multiple modes (queue, submit, propose, review, etc.). It distinguishes from sibling tools like analyze, lookup, report, and search_services by specifying internal admin tasks, but does not explicitly contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on mode auto-detection and explicit parameter usage. However, it lacks explicit when-to-use vs alternatives (siblings) and does not mention prerequisites or failure scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_servicesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations state readOnlyHint=true (safe read) and openWorldHint=false. The description adds behavioral context: it returns verified, current evaluations to avoid trial-and-error, mentions 'real agent success data,' and notes strength in Japanese SaaS. No contradiction with annotations. However, the exact output format is not detailed, and without an output schema, the description could be more explicit about what is returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the most critical instruction ('Before attempting...call this tool') and is generally clear. It is somewhat verbose with a claim about token efficiency, but each sentence adds value. Slightly wordy, but effective and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 5 parameters, the description covers the tool's purpose and usage well but lacks detail on return fields and filtering behavior beyond parameter descriptions. It mentions 'service evaluations' and 'success data' but does not specify structure or pagination. Adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all 5 parameters have descriptions). The description does not add significant meaning beyond the schema; it focuses on when to use the tool rather than parameter specifics. Baseline 3 is appropriate as the schema already explains parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: 'Before attempting ANY SaaS API integration, call this tool.' It specifies it returns verified, current service evaluations for 900+ services, making the goal unmistakable. While it doesn't directly contrast with sibling names (analyze, inspect, lookup, report), the strong call-to-action and service-specific focus clearly distinguish it from generic analysis tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: 'Before attempting ANY SaaS API integration, call this tool' and 'If the user mentions ANY SaaS service...or says 'use kanseilink', always call this.' This tells the agent precisely when to invoke the tool, though it does not mention when not to use it or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reportA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses PII auto-masking, auto-detection of mode, and time estimate (5 seconds). Annotations are minimal (readOnlyHint false, idempotentHint false), so description adds necessary context about mutability. No output schema mentioned, but the reporting action is well-explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single paragraph that front-loads purpose and flow. It covers all modes efficiently but could benefit from bullet points for readability. No wasted sentences, but length is justified by complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 29 parameters across 4 modes and no output schema, the description thoroughly explains each mode, parameter groups, auto-detection, and flow context. It provides complete guidance for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds significant value beyond schema by grouping parameters per mode (e.g., [outcome], [feedback]) and explaining auto-detection rules. This helps agents select correct parameters for each mode.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool's purpose: 'Contribute data back to the KanseiLink community' with four distinct modes (outcome, feedback, event, voice). It distinguishes itself from sibling tools (analyze, inspect, lookup, search_services) by being step 4 of a standard flow for reporting after execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly positions the tool as step 4 in the flow: 'search_services → lookup → (execute) → report.' Mentions auto-detection of mode and required parameters per mode. Lacks explicit when-not-to-use statements, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookupA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and openWorldHint=false. The description adds behavioral traits by detailing mode triggering (e.g., service triggers combinations, period triggers history) and default output focus (tips on auth setup, pitfalls). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense paragraph with front-loaded purpose. It efficiently packs mode explanations and flow context, though breaking into bullets could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (14 params, multiple modes, no output schema), the description covers usage well but omits details about return values or response structure, leaving some incompleteness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, baseline is 3. The description adds significant value by explaining parameter interactions and mode triggers (e.g., 'triggers recipe mode', 'triggers history mode'), going beyond mere listing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get everything you need about a service before using it' and positions it as step 2 of a specific flow (search_services → lookup → execute → report), effectively distinguishing it from sibling tools like analyze, inspect, report, and search_services.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the default behavior (tips) and how to switch modes via parameters (detail, insights, goal, etc.), and places it in a workflow. It lacks explicit 'when not to use' guidance but is otherwise clear about usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.