Server Quality Checklist

Tool Scores

• claude_promptA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: stateless operation, no session persistence, no resume capability, and that it uses the server's current working directory. However, it doesn't mention rate limits, authentication requirements, error handling, or response format details that would be helpful for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place. The first sentence states the core functionality and key behavioral constraints. The second sentence provides important operational context about the working directory. There's zero wasted language or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description provides adequate but incomplete context. It covers the operational mode and working directory context well, but doesn't address what the return value looks like (only says 'Returns the model's text response' without format details), error conditions, or performance characteristics that would help an agent use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all three parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema - it doesn't explain parameter interactions, provide examples, or clarify usage patterns. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Run a one-shot prompt'), target resource ('Claude Code CLI'), and operational mode ('headless, stateless mode with no session persistence, no resume'). It distinguishes from siblings by specifying this is a one-shot operation without persistence, unlike tools that might maintain context or structure outputs differently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('headless, stateless mode with no session persistence, no resume'), which implicitly suggests alternatives when persistence or session management is needed. However, it doesn't explicitly name sibling tools or provide explicit 'when-not-to-use' guidance beyond the stateless nature.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• claude_prompt_with_contextA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses important behavioral traits: 'headless, stateless mode', 'Returns the model's text response', and 'Uses the server process's current working directory; file paths are resolved relative to cwd.' However, it lacks details about rate limits, authentication needs, error conditions, or response format specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core functionality and key differentiators, the second adds important operational context about working directory and file path resolution. Every element earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and no output schema, the description provides adequate but not complete coverage. It explains the core operation and some behavioral context but lacks details about response format, error handling, or model selection implications. For a tool with this complexity and no structured safety/behavior annotations, more completeness would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds some context about how 'context' and 'files' parameters work ('prepended to the prompt'), but doesn't provide significant additional meaning beyond what's in the schema descriptions. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run a one-shot prompt'), the target ('Claude Code CLI'), the mode ('headless, stateless mode'), and key differentiators from siblings ('with additional free-form context and/or file contents prepended'). It distinguishes this tool from 'claude_prompt' and 'claude_prompt_structured' by emphasizing the context/file prepending capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('with additional free-form context and/or file contents prepended to the prompt'), which implicitly suggests alternatives when such context isn't needed. However, it doesn't explicitly state when NOT to use it or name specific sibling alternatives beyond the general differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• claude_prompt_structuredA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key traits: the tool runs in 'headless, stateless mode', validates output against a schema if provided, returns parsed JSON, and uses the 'server process's current working directory'. This covers execution mode, validation behavior, and environmental context, though it lacks details on error handling or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first. Every sentence adds value: the first defines the action and output, the second explains schema usage, and the third provides environmental context. There is no wasted text, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a 4-parameter tool with no annotations and no output schema, the description is mostly complete. It covers the tool's purpose, behavioral traits, and usage context. However, it lacks details on error cases, response format beyond 'parsed JSON', or performance considerations, leaving minor gaps for a tool with no structured safety hints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal parameter semantics beyond the schema, only mentioning that the schema is 'used both as an instruction to the model and for lightweight post-hoc validation'. This aligns with the baseline score of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run a one-shot prompt against the Claude Code CLI in headless, stateless mode and return a structured JSON object.' It specifies the verb ('Run'), resource ('Claude Code CLI'), and distinguishes from siblings by mentioning 'structured JSON object' and schema validation, unlike the generic 'claude_prompt' and context-aware 'claude_prompt_with_context'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for 'one-shot prompt' execution in 'headless, stateless mode' with structured JSON output. It implies usage for schema-constrained responses but does not explicitly state when NOT to use it or name alternatives like the sibling tools, though the structured output focus differentiates it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.