Server Quality Checklist

Tool Scores

• trigger_auto_recoveryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'automatic recovery' and 'predefined playbooks,' but does not disclose critical behavioral traits such as whether this is a read-only or destructive operation, what permissions are required, potential side effects, or how recovery is executed. This is inadequate for a tool that likely involves system changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('trigger automatic recovery') without unnecessary words. Every part earns its place by specifying the target and mechanism concisely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a recovery tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral aspects like safety, permissions, or what happens post-recovery, which are crucial for an agent to use this tool correctly in an infrastructure context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the two parameters. The description does not add any meaning beyond the schema, such as explaining how 'service_name' and 'failure_type' interact with playbooks. Baseline 3 is appropriate as the schema handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('trigger automatic recovery') and target ('for a failed service'), specifying the mechanism ('using predefined playbooks'). It distinguishes from siblings like 'restart_service' by emphasizing automation and playbooks, though it could be more specific about what 'automatic recovery' entails compared to manual alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a service has failed, based on 'failed service,' but does not explicitly state when to use this tool versus alternatives like 'restart_service' or 'check_services_health.' No exclusions or prerequisites are mentioned, leaving usage context somewhat vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_services_healthB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Safe read-only operation,' which is valuable context about safety and operation type. However, it doesn't describe what 'health' means (e.g., metrics, status codes), whether it requires authentication, rate limits, or what the output looks like (e.g., structured data, alerts).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences that earn their place: the first states the purpose, and the second adds critical behavioral context ('Safe read-only operation'). It's front-loaded with the core function and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is minimally complete for a read-only health check tool. It covers the basic purpose and safety but lacks details on output format, error handling, or integration with sibling tools. For a tool with 2 parameters and moderate complexity, it should ideally explain what 'health' entails or link to related tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for both parameters ('service_group' with enum-like values and 'service_names' as an optional array). The description adds no additional parameter semantics beyond what's in the schema, such as explaining how parameters interact (e.g., if 'service_names' overrides 'service_group'). Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Check health of one or more services' with the verb 'Check' and resource 'services'. It distinguishes from siblings like 'restart_service' (which modifies) and 'get_infrastructure_status' (which is status-focused rather than health-focused). However, it doesn't explicitly differentiate from 'trigger_auto_recovery' which might also involve health assessment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to use 'check_services_health' instead of 'get_infrastructure_status' (which might provide overlapping information) or 'trigger_auto_recovery' (which might be for automated responses to health issues). No context about prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_infrastructure_statusA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Safe read-only operation', which clarifies that it's non-destructive and safe to use. However, it doesn't mention other behavioral aspects like performance implications, rate limits, or authentication needs, which would be helpful for a comprehensive infrastructure tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that are front-loaded and waste no words. The first sentence states the purpose, and the second adds critical behavioral context, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (infrastructure overview) and lack of annotations or output schema, the description is minimally adequate. It covers safety and purpose but doesn't explain what 'comprehensive overview' entails in terms of return values or structure, leaving gaps that the agent must infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already fully documents the two parameters (include_history and include_locks). The description adds no additional parameter semantics beyond what the schema provides, making the baseline score of 3 appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('comprehensive overview of entire infrastructure'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'check_services_health', which might overlap in functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_services_health' or 'trigger_auto_recovery'. It lacks context about specific scenarios, prerequisites, or exclusions, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_cloudflare_workerA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'state tracking and health checks,' which adds some context beyond the basic deployment action, but lacks details on permissions, rate limits, error handling, or what 'state tracking' entails, leaving significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and includes a critical usage guideline in just two sentences, with zero wasted words, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a deployment tool with no annotations and no output schema, the description is minimally adequate. It covers the purpose and usage but lacks details on behavioral aspects like error handling or return values, leaving room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description does not add any additional meaning or context about the parameters beyond what the schema provides, resulting in the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Deploy Cloudflare Worker') and resource ('Cloudflare Worker'), making the purpose evident. However, it doesn't differentiate from sibling tools like 'deploy_mcp_server' beyond the specific resource type, which keeps it from a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance by stating 'DO NOT use ssh_exec('wrangler deploy') - always use this tool instead,' which clearly indicates when to use this tool versus an alternative method, though it doesn't compare to sibling tools directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_mcp_serverA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'state tracking, locks, and health checks,' which adds useful context about deployment behavior beyond just the action. However, it lacks details on permissions, error handling, or what 'deploy' entails operationally, leaving gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with two sentences that efficiently convey the purpose and critical usage rule. Every sentence adds value without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a deployment tool with no annotations and no output schema, the description provides basic purpose and usage rules but lacks details on what 'deploy' entails (e.g., steps, rollback), expected outcomes, or error scenarios. It's minimally adequate but has clear gaps given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description doesn't add any parameter-specific information beyond what's in the schema, such as explaining interactions between parameters or additional constraints. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Deploy MCP server to Fly.io') and resource ('MCP server'), making the purpose evident. However, it doesn't explicitly differentiate from sibling tools like 'deploy_cloudflare_worker' beyond the target platform, missing specific distinctions about when to choose one over the other.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'DO NOT use ssh_exec for deployments - always use this tool instead.' This clearly states when to use this tool (for deployments) and when not to use an alternative (ssh_exec), offering strong contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restart_serviceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the tool performs a restart operation (implying mutation), includes 'pre/post health checks' (safety mechanisms), and has 'automatic rollback' (error handling). However, it doesn't specify permission requirements, rate limits, or what constitutes successful health checks.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that each serve clear purposes: the first states the core functionality with key behavioral details, and the second provides critical usage guidance. There's no wasted language or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description does well by explaining the restart operation, safety features (health checks, rollback), and usage rules. However, it doesn't specify what the tool returns or error conditions, leaving some gaps in completeness for an agent invoking this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for adequate but not enhanced parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Restart a service') and distinguishes it from a prohibited alternative ('DO NOT use ssh_exec('docker restart')'), making the purpose explicit and differentiated from potential siblings. It goes beyond just naming the action by specifying the method includes 'pre/post health checks and automatic rollback'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('always use this tool instead') and when not to use alternatives ('DO NOT use ssh_exec('docker restart')'), with a clear directive that establishes this as the preferred method for service restarts in this context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.