Server Quality Checklist

Tool Scores

• caddy_config_deleteA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the operation as destructive and not read-only. The description usefully adds that deleting a parent node cascades to all descendants and provides a concrete example, plus the confirm=true safety guard. This goes beyond the structured annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words: the core action is first, then the critical cascade behavior with an example, then the required confirmation flag. Each sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive delete operation, the description covers the target path, the example, cascade semantics, and the safety gate. It omits details like behavior for nonexistent paths or response/error values, but annotations and schema already cover the key safety profile.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and both parameters already have descriptive documentation in the schema. The description repeats the confirm requirement and gives an example path, adding little new meaning about the parameters themselves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete config at a JSON path') and the resource (Caddy config), with a precise definition of deleting a node and its descendants. It does not explicitly differentiate from sibling tools like caddy_remove_route, but the JSON-path framing makes the scope reasonably distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to choose this tool over alternatives such as caddy_config_set or caddy_remove_route. The confirm=true requirement is a safety parameter, not a usage-or-alternative guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_pkiA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds little beyond repeating the certificates parameter's effect, and does not disclose additional behavioral traits such as output format or error behavior. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that begins with the verb and directly states the tool's purpose. No filler or redundant information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple read-only getter with no output schema, yet the description is vague about what 'info' includes or what the response format looks like. While annotations and schema cover parameters and safety, the description lacks contextual detail about the return value or edge cases, making it minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% (both 'ca' and 'certificates' have descriptions). The description does not add any additional meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with the verb 'Get' and clearly identifies the resource as 'PKI certificate authority info or the CA certificate chain.' This is specific and distinguishes it from siblings like caddy_config_get or caddy_status, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its usage as a read-only query for PKI CA information, but does not explicitly state when to use it over alternatives like caddy_tls or caddy_config_get. No exclusions or alternative tool references are provided, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_add_routeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false, providing minimal safety context (not read-only, not idempotent, not destructive). The description adds capability details (supports any handler) but does not disclose behavioral traits such as whether the route is appended or overwrites existing routes, validation behavior, or reversibility. This is a moderate gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and contains no fluff. Every word adds relevant context about what the tool does and its flexibility.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 100% schema coverage and absence of an output schema, the description covers the tool's purpose and scope effectively. It misses some behavioral context (e.g., route ordering, interaction with terminal flag) but is reasonably complete for a config-modifying tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by enumerating common handler types (reverse_proxy, file_server, etc.) and emphasizing 'full control', which helps the agent understand the flexible, free-form nature of the match and handle parameters beyond the schema's minimal examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Add a route') and its scope ('full control over match conditions and handlers'). It also lists specific handler types, which distinguishes it from sibling tools like caddy_remove_route and caddy_list_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'full control' implies this is the general-purpose tool for adding routes, and mentioning 'any Caddy handler' suggests flexibility. However, it does not explicitly state when to use this versus alternatives like caddy_reverse_proxy, nor does it provide any exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_stopA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false. The description adds meaningful context by specifying 'gracefully' shutdown and the confirm=true safeguard, which helps the agent understand the manner and safety requirement of the operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core action, and every word contributes value. There is no filler or redundant elaboration beyond the necessary confirmation note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter stop operation with clear annotations and full schema coverage, the description is sufficient. It does not explain return values or edge cases, but given the low complexity and existing structured data, that is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the confirm parameter with 100% coverage ('Must be true to confirm shutdown'). The description only repeats this requirement without adding new semantic detail, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'shut down' with a clear resource 'Caddy server', and it is distinct from sibling tools that handle config, status, or routes. The action and target are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use (to stop the server) but does not explicitly state when to use it versus alternatives or when not to use it. The confirm requirement is a prerequisite, not a usage guideline. It lacks explicit context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_by_idA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already flag destructiveness (destructiveHint=true, readOnlyHint=false), so the bar is lower. The description adds concrete safety behavior beyond the annotations: 'The delete action requires confirm=true,' and it clarifies the tool is multi-modal (read/update/delete), which nuances the blanket destructive hint by showing only one of three actions is destructive. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler: the mechanism, the scope of operations, and the one safety caveat that must not be missed (confirm=true). The key concept (addressing by @id) is front-loaded, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a six-parameter tool with no output schema, the description covers the conceptual model (what @id is), the available actions, and the delete safeguard, while the 100%-coverage schema handles all parameter detail. The only gaps are response shape and error behavior for unknown @ids, which are secondary for tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the mode enum already documenting PATCH/POST/PUT semantics, so the baseline is 3. The description adds useful conceptual context for the id parameter — that any config object carrying an '@id' field is addressable — but it adds nothing about value, subpath, action, or confirm beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific mechanism and resource ('Access config by @id tag') and expands to enumerate the three operations ('read, updated, or deleted') and the addressing advantage ('instead of needing its full path'). This substantively distinguishes the tool from the path-based caddy_config_get/set/delete siblings by its unique addressing model, even though it never names them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'This is the recommended way to manage individual routes and config objects' gives an explicit positive recommendation for when to use the tool, and 'instead of needing its full path' implies the alternative of path-based config access. However, it names no sibling tool explicitly and gives no exclusion criteria, leaving the boundary against caddy_add_route/remove_route implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_getA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds valuable behavioral context by explaining the return behavior for empty vs. specific paths, which goes beyond the structured annotations and helps the agent predict the tool's output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every word adds value. It avoids needless detail while including a practical example. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one optional parameter and no output schema, the description covers the essential behaviors: full config vs. subtree, path example, and empty path default. It doesn't mention error handling or exact output format, but these are relatively minor gaps given the tool's simplicity and strong annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the only parameter 'path', but the description enriches its meaning by explicitly stating that an empty path returns the full config and providing a concrete example ('apps/http/servers/srv0/routes'). This goes beyond the schema's generic 'Config path' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads Caddy config at a JSON path, with a specific verb ('Read') and resource. It distinguishes itself from siblings by being the generic config getter, while other tools handle writes, deletes, or specific lookups. The clarification of empty path returning the full config adds scope precision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended usage is implied: use this to read config by path. However, there is no explicit guidance on when to prefer this tool over alternatives like caddy_config_by_id or caddy_status, nor any exclusions. The description provides clear context but no comparative direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_upstreamsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds value beyond annotations by explicitly listing the output fields (address, active requests, failure counts), which is the primary behavioral detail for a status tool. It does not contradict annotations and provides useful context about what the agent will receive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately stating the action and resource, then enumerating the returned data. Every word is functional with no filler or repetition. It is front-loaded with 'Get the current health status' and remains compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema, strong annotations), the description covers the essential aspects: what it does and what it returns. It lists the three key fields, which is sufficient for an agent to understand the tool's output. However, it does not specify the exact response format or how health status is determined, leaving minor ambiguity for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty with 0 parameters, so the baseline is 4. The description correctly makes no mention of parameters, as none exist. It neither adds nor needs to add parameter semantics since there is nothing to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific function: getting current health status of reverse proxy upstreams. It lists the exact data shown (address, active requests, failure counts), distinguishing it from sibling tools like caddy_status (overall Caddy status) and caddy_reverse_proxy (likely configuration). The verb 'Get' plus the specific resource makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need upstream health information) but provides no explicit guidance on alternative tools or exclusions. It does not mention when not to use it or how it compares to caddy_status or caddy_metrics, leaving the agent to infer usage context from the purpose statement alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_revertA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare destructiveHint=true, and the description aligns by noting 'apply' restores a snapshot and requires confirm=true. It adds valuable context beyond annotations: snapshots are auto-captured before caddy_load, are kept to the last 10, and are lost on server restarts unless a directory is configured. This discloses persistence and destructive implications that annotations alone do not convey. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and each clause earns its place. It packs a lot of information—auto-capture, retention, persistence caveat, and actions—without bloat. The structure is efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters and no output schema, the description covers the essential behavior: what actions exist, persistence, and the safety requirement for apply. It doesn't describe return types (not required without output schema) or error handling, but those are minor. It could go further by explicitly stating that 'apply' overwrites the active config, but that is strongly implied by 'restores a snapshot'. Overall it is sufficiently complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter is documented in the schema. The description nevertheless adds semantic value by explaining the meaning of each action value ('list' shows snapshots, 'save' captures, 'apply' restores), and reinforcing that confirm must be true for apply. While the schema already describes index as '0 = most recent', the description's action semantics help the agent understand the workflow without opening an enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool manages config snapshots for rollback, and enumerates three specific actions (list, save, apply). This distinguishes it from sibling tools like caddy_load or caddy_config_set, which deal with live config rather than snapshots. The verb 'manage' plus the resource 'config snapshots' is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (for snapshot operations) and includes a critical caveat about snapshots being memory-only and lost on restart unless CADDY_MCP_SNAPSHOT_DIR is set. It also notes that 'apply' requires confirm=true, which is a usage requirement. However, it does not explicitly contrast with alternatives like caddy_load or caddy_config_set, leaving some routing to inference. The context is clear but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_adaptA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false. The description adds behavioral context: conversion is done locally ('without loading it'), returns 'adapted JSON and any warnings separately', and requires adapter modules to be compiled in. This exceeds annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the main verb 'Convert', with each sentence providing necessary detail (purpose, usage, output). No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description covers purpose, usage, prerequisites, and return format. It is self-contained for an agent to decide when to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions. The description enriches by listing built-in ('caddyfile') and external adapters ('nginx', 'yaml') and noting adapter module requirements, which adds meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'Convert a config in any registered adapter format to Caddy JSON without loading it.' This clearly identifies the action (convert), the resource (config), and the output (Caddy JSON). The 'without loading it' distinguishes it from loading-related siblings like caddy_load.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Useful for previewing what a Caddyfile produces, or for porting from nginx/yaml configs.' This gives explicit use cases. However, it does not explicitly name sibling alternatives or state when not to use, so it provides clear context but no exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_list_serversA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare this as a safe, read-only, idempotent operation. The description adds transparency about the output content (specific server attributes), which is helpful given there is no output schema. This goes beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the action and output details, followed by a practical usage hint. Every sentence contributes value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list operation with strong annotations, this description is complete. It tells the agent exactly what the tool does, what it returns, and when to invoke it, making tool selection and usage straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is fully covered by default. Per the baseline for 0-parameter tools, the description appropriately focuses on behavior and output rather than parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the operation as listing all configured HTTP servers and specifies the exact data returned (names, listen addresses, route counts, TLS status). This distinguishes it from sibling tools like caddy_list_routes, which focus on routes rather than servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this to discover server names before calling route tools.' This provides a clear workflow context, though it does not explicitly name alternatives or exclusion cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe read-only nature is clear. The description adds valuable context about what the summary contains (servers, routes, listen addresses, TLS status), going beyond the annotations. It does not discuss failure modes, but the annotation coverage lowers the bar and the added context is meaningful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that packs significant information: the verb ('Check'), the resource ('Caddy connectivity'), and a detailed summary content list. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description is complete. It explains both the action (connectivity check) and the expected return content (config summary with servers, routes, listen addresses, TLS status). Rich annotations further cover the safety profile, so the description sufficiently covers all necessary contextual information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so the schema is trivially complete (coverage 100%). Per guidelines, 0 params receives a baseline score of 4 because there are no parameter gaps to address. The description adds no parameter info, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Check Caddy connectivity and get a config summary'. It also enumerates the summary content (servers, routes, listen addresses, TLS status), which distinguishes it from sibling tools like caddy_config_get (full config) and caddy_list_servers (specific server list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool provides connectivity checks and a config summary, but it does not explicitly recommend it over alternatives or mention when not to use it. It provides clear context for when this tool is appropriate, but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_setA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes far beyond the sparse annotations by detailing HTTP semantics (PATCH/POST/PUT), idempotency per mode, and side effects like route duplication on repeated 'append' calls. It also highlights the 'insert' mode's utility for ordering. This fully discloses behavioral traits and potential pitfalls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (two sentences) and front-loaded with the core purpose before detailing modes. Each mode is explained with relevant consequences, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's behavior comprehensively: modes, idempotency, side effects, and ordering use-case. It doesn't mention return values, but the absence of an output schema makes that less critical. Given the complexity of the tool (three modes with different behaviors), this description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already well-documented. The description adds value by explaining the mode-specific behaviors (idempotency, side effects) that the schema enum descriptions only hint at. It does not add much for 'path' or 'value', but the schema already provides examples and definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Write config at a JSON path.' It specifies the resource (config), the action (write), and the path-based scope. This distinguishes it from sibling tools like caddy_config_get or caddy_config_delete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use each mode: 'overwrite' for safe/idempotent replacement, 'append' for adding to arrays (with warning about duplication), and 'insert' for route ordering. It doesn't explicitly mention alternatives to the tool itself, but the mode guidance is strong and context-rich.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_list_routesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly/idempotent/non-destructive, and the description adds significant behavioral detail: output caps (500 routes/20000 chars), whole-route truncation, omission note, and exact fallback path. This fully discloses edge-case behavior without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences that each add distinct value: purpose/output, caps/truncation, and fallback. No redundant filler; structure is well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation with one optional parameter and no output schema, the description covers output format, limits, edge-case behavior, and how to get the complete data. It is fully self-contained for an agent to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'server' is already documented in the schema with default 'srv0' and 100% coverage. The description does not add additional parameter-level detail but doesn't need to given the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists all routes on a Caddy HTTP server and describes the dual output (summary + raw JSON). The use of 'all routes' and explicit reference to caddy_config_get for reading the rest distinguishes it from sibling config-read tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when you want a quick route overview on a specific server, and explicitly directs to caddy_config_get at a specific JSON path for the full untruncated data. The cap behavior and omission note also tell the agent when the result is incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_loadA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations already declaring destructiveHint and idempotentHint, the description adds significant behavioral detail: atomicity, a 60-second timeout for TLS provisioning, the mandatory confirm=true requirement, the fact that the entire running config is discarded, and that a snapshot is taken allowing restore via caddy_revert. These details greatly exceed what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, with the primary action front-loaded in the first sentence. Every sentence carries substantive information: format options, safety/timeout, destructive consequences, and snapshot/restore. There is no filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive, high-stakes operation, the description provides comprehensive guidance: it warns about the full config discard, mandates confirm=true, mentions atomicity and the timeout, and explains the snapshot/revert path. It also positions itself relative to large config changes. The absence of an output schema is not a meaningful gap since the return value is secondary to the behavioral warnings covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage by describing all three parameters, but the description adds complementary meaning: it clarifies that config can be a JSON object or a Caddyfile string, ties format='caddyfile' to that choice, and explains that confirm=true is required because the action discards the entire running config. This adds context beyond the schema, though it does not individually explain every parameter in depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Replace the entire Caddy configuration atomically,' using a specific verb ('replace') and a clear resource ('entire Caddy configuration') with a distinctive behavior ('atomically'). This distinguishes it from sibling tools like caddy_config_set or caddy_config_delete, which likely target partial changes. The mention of accepting JSON or Caddyfile further clarifies scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'This is the safest way to make large config changes,' providing clear context for when to use this tool. It also mentions restoration via caddy_revert, giving an alternative fallback. However, it does not explicitly say when not to use it (e.g., for small targeted changes) or name alternative tools like caddy_config_set, so it stops short of a full when/when-not distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_remove_routeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by clarifying that idempotency is mode-dependent: @id mode is idempotent, while index mode is not and can remove different routes on repeated calls. It also discloses the two-step read-then-delete race, the config-global scope of @ids, and the refusal behavior when the id resolves to a non-route object. This is excellent behavioral disclosure for a destructive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence carries essential operational or safety information. The description is front-loaded with the core action, then systematically covers target selection, idempotency differences, concurrency hazards, and scope semantics. There is no filler or repetition that does not earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive, non-idempotent tool with no output schema, the description is remarkably complete. It addresses the main failure modes (race conditions, repeated index removal, non-route @id targets), explains the confirm-based safety mechanism indirectly through the schema, and gives the agent enough context to call the tool correctly and avoid destructive mistakes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema: it explains why id is preferred, warns that index removal re-packs the routes array, and clarifies that @ids are config-global rather than route-scoped. This enriches the agent's understanding of how id and index actually behave.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a direct, specific statement—'Remove a route'—and immediately identifies the two targeting modes (@id or index on a specific server). This clearly distinguishes it from generic config deletion tools and from route-modification siblings like caddy_add_route or caddy_list_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: prefer @id because it is stable across reorderings, avoid index mode when possible because it can race against concurrent edits. It does not explicitly compare this tool to alternatives like caddy_config_delete, but the contextual guidance for choosing between the tool's own modes is strong and concrete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_metricsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds substantial behavioral context: output can be megabytes, filter-mode drops blank/free-form comments but preserves HELP/TYPE lines, '# EOF' is always kept, max_lines defaults to 500, and a trailing comment reports dropped lines. This goes well beyond the structured hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized—four sentences that lead with the core purpose, then dive into parameter behavior with examples. Every sentence carries unique, essential information; there is no repetition or filler. The structure flows logically from purpose to usage to edge cases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description covers the return format implicitly (Prometheus text) and explicitly covers filtering semantics, line handling, size limits, and the trailing summary. It gives the agent enough context to anticipate output characteristics and pitfalls, making it complete for invocation and result handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description significantly enriches parameter meaning. It explains that filter is a substring match on metric names, that HELP/TYPE lines for matching metrics are retained, and that label values are NOT matched. For max_lines, it details the default (500) and the appended summary line, which the schema does not fully convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Get Prometheus metrics from Caddy.' It lists concrete metric categories (request counts, durations, TLS handshake stats, active connections) and clearly distinguishes this tool from sibling tools like caddy_config_get or caddy_status, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the filter and max_lines parameters, including examples and edge-case behavior (label values not matched). It does not explicitly contrast with alternatives, but among the siblings none offer metrics, so the context is clear enough for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_reverse_proxyA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses conditional idempotency (append vs PATCH replace), config-global @id semantics, collision refusal, behavior after an id is registered, and TLS transport rules. This goes well beyond the coarse annotations and is not contradicted by them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Dense but well organized: the one-line purpose and example come first, followed by idempotency, id-scoping behavior, and upstream handling. Every sentence carries operational information; there is no filler or repetition beyond what is safety-relevant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a config-mutating tool with four parameters and no output schema, this description covers all calling modes, failure modes (collision refusal, mixed-scheme refusal), and side effects (in-place replacement, global id behavior). An agent has enough information to call it correctly and predict repeat-call behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds operational meaning beyond the schema: id omission vs supply changes idempotency, upstream scheme changes transport and default port, and mixed scheme lists are refused. The example also makes the from/to shape concrete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Add a reverse proxy route') and gives a concrete from/to example, making the operation unmistakable. The domain-specific phrasing also separates it from the generic sibling caddy_add_route, and the final note references caddy_add_route as the lower-level fallback.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends supplying a stable id for automation, warns that omitting id duplicates routes, and gives a clear alternative (caddy_add_route) for mixed https/non-https upstreams. This gives an agent concrete decision rules rather than leaving tool selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_tlsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare idempotentHint=true and destructiveHint=false, and the description adds substantial behavioral context: write scope is restricted to policies[0].issuers[0], writes only occur when the issuer module is 'acme', 'ech_status' is read-only, and profile validity is CA-defined. This fully discloses side effects and limitations beyond the annotation hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: actions are front-loaded, followed by compatibility notes, write-scope constraints, and a concrete routing instruction to a sibling tool. No filler or restatement of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of five actions with different constraints, the description fully covers behavior, version requirements, write targets, and fallback routing to caddy_config_set. The absence of an output schema is mitigated by the clear action-level descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters, but the description adds meaning beyond it: the profile parameter's version requirement, CA-defined valid values, and the Let's Encrypt 'shortlived' example. This enriches the agent's understanding without redundant repetition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description leads with a clear verb-resource pairing ('Get or configure TLS/HTTPS settings') and enumerates every supported action with specific targets (ACME email, CA URL, profile, ECH status). It distinguishes itself from caddy_config_set by naming the exact condition under which that sibling should be used instead.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use and when-not-to-use guidance: it works on fresh or existing instances, but for multi-policy TLS configs or non-ACME first issuers it directs the agent to use caddy_config_set. It also flags version-dependent actions (Caddy 2.10+), leaving no ambiguity about applicability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.