Server Quality Checklist

Tool Scores

• searchA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds useful context beyond annotations: cost estimates, recommendation to increase timeout for complex queries, and that results include source URLs. However, it does not clarify the non-read-only nature (readOnlyHint=false) or any side effects, leaving some behavioral traits implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a clear opening sentence, bullet points for use cases and tips, and a cost note. It is front-loaded with purpose and well-structured, with no redundant sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 9 parameters and no output schema, the description covers purpose, use cases, cost, and practical tips. It could mention result pagination or format details, but overall it provides sufficient context for an AI agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 9 parameters have schema descriptions (100% coverage). The description adds practical usage guidance for maxResponseLength, timeout, and effort, helping the agent understand how to use them effectively beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs web search via Claude Code CLI and synthesizes a comprehensive answer with source URLs. It is distinct from sibling tools like listSessions, ping, query, and structured, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists use cases (e.g., current information, documentation lookups) and provides tips (e.g., ask specific questions, use maxResponseLength). It does not specify when not to use or compare directly with alternatives, but the context is clear enough for appropriate selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listSessionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal readOnly and idempotent. Description adds 'No cost (local lookup only)', clarifying behavior beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no redundant information. Each sentence contributes unique value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, return content, usage context, and cost. No output schema, so description fully compensates. All necessary information present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Baseline 3 applies; description adds value by listing return fields but not required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List active Claude CLI sessions' with specific resource and verb. It distinguishes from siblings like search and query by focusing on sessions tracked by this server, making it unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit usage guidance: 'Use to check available sessions before resuming with sessionId.' Does not state when not to use, but context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• structuredA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations (readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true) provide basic safety signals. The description adds valuable behavioral context: it uses native CLI validation (not client-side), cost is ~$0.01-0.10/call and unaffected by schema complexity, and it is designed for machine-parseable output. These details go beyond what annotations convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a lead sentence stating the core function, then a brief use case bullet, a cost note, and a tips section. Every sentence serves a purpose with no redundancy. The structure is front-loaded with the most important information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no output schema, and moderate complexity, the description covers the essential aspects: purpose, use cases, cost, and parameter tips. It could mention default behavior (e.g., timeout default of 60000) or error handling, but the parameter schema covers most details. Overall, it is sufficiently complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 9 parameters have descriptions in the schema (100% coverage), so baseline is 3. The description adds extra meaning: for the 'schema' parameter it notes 'Pass the JSON Schema as a JSON string' and 'Schema max size: 20KB'; for 'files' it explains 'include source text via the files parameter or inline in the prompt.' These tips improve parameter understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Generate JSON conforming to a provided JSON Schema' using a specific implementation (Claude CLI's --json-schema flag). It lists concrete use cases like data extraction, classification, and entity parsing, which distinguish it from sibling tools such as 'query' (likely free-form) and 'search' (retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for: data extraction from text/files, classification, entity parsing, or any task needing machine-parseable output.' It also provides practical tips (schema max size, including source text via files parameter) but does not explicitly state when not to use this tool or mention alternatives beyond implying it's better than client-side validation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent. Description adds that it's local only and costless, providing extra behavioral context beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the purpose and include all necessary details without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, rich annotations, and no output schema, the description fully covers purpose, scope, and cost, making it complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Baseline for zero parameters is 4, and no additional parameter info needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool performs a health check for Claude CLI, verifying installation, authentication, and reporting versions, capabilities, configuration. It is specific and distinguishes itself from sibling tools that handle data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly notes 'No cost (local check only)', implying when to use (verify setup) and when not (no remote calls). While it doesn't explicitly compare to siblings, the purpose is distinct enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate non-destructive and non-idempotent behavior with open-world hint. The description adds context: capabilities, cost structure, session management, and limitations like image query timeout. No contradiction with annotations; the tool's potential to generate/refactor code is noted but not claimed to modify persistent data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly lengthy but front-loaded with the core purpose, followed by capabilities, cost, and tips. It is well-structured but could be slightly more concise without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, no output schema), the description is comprehensive: covers purpose, usage, parameter semantics, cost, and practical tips. It adequately equips an agent to understand when and how to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description adds valuable meaning beyond each parameter's schema: explains files as text/images, sessionId for resume, maxBudgetUsd as cost cap, effort levels, and workingDirectory for context. Tips further clarify parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute a prompt via Claude Code CLI' and lists capabilities like code generation, analysis, and multi-turn conversations. It distinguishes itself from sibling tools (listSessions, ping, search, structured) by focusing on prompt execution with optional file context and session resume.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive usage guidance: cost info, effort levels for different task complexities, tips for workingDirectory, breaking tasks, resuming sessions, including files, and stateless calls. However, it does not explicitly compare this tool to siblings (e.g., when to use search instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.