Server Quality Checklist

Tool Scores

• caddy_pkiA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds little beyond repeating the certificates parameter's effect, and does not disclose additional behavioral traits such as output format or error behavior. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that begins with the verb and directly states the tool's purpose. No filler or redundant information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple read-only getter with no output schema, yet the description is vague about what 'info' includes or what the response format looks like. While annotations and schema cover parameters and safety, the description lacks contextual detail about the return value or edge cases, making it minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% (both 'ca' and 'certificates' have descriptions). The description does not add any additional meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with the verb 'Get' and clearly identifies the resource as 'PKI certificate authority info or the CA certificate chain.' This is specific and distinguishes it from siblings like caddy_config_get or caddy_status, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its usage as a read-only query for PKI CA information, but does not explicitly state when to use it over alternatives like caddy_tls or caddy_config_get. No exclusions or alternative tool references are provided, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_tlsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotent behavior and non-destructiveness. The description adds minimal behavioral context beyond the actions themselves, such as working on fresh/existing instances. It does not contradict annotations, and while it does not provide deeper details like side effects or permission requirements, the annotations reduce the burden. The added info is useful but limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise at two sentences. It front-loads the primary purpose, then lists actions with brief clarifications. Every sentence adds value, with no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While there is no output schema, the tool's scope is limited and clearly described. The actions and their parameter mappings are covered by the description and schema, and the annotations cover safety concerns. The only minor gap is not describing the return format of 'status', but overall completeness is strong for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter description coverage, clearly defining each parameter and its association with an action. The description essentially restates the action enum but adds no deeper syntactic or formatting details beyond what the schema provides. Since schema coverage is complete, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function ('Get or configure TLS/HTTPS settings') and enumerates specific actions with their meanings. It distinguishes itself from sibling tools like caddy_config_get or caddy_status by focusing specifically on TLS/HTTPS settings and ACME configuration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some context by stating it works on both fresh and existing Caddy instances, but it does not explicitly state when to prefer this over related tools (e.g., caddy_config_get/set) or specify conditions under which it should not be used. No alternatives are mentioned, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_add_routeA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false, providing minimal safety context (not read-only, not idempotent, not destructive). The description adds capability details (supports any handler) but does not disclose behavioral traits such as whether the route is appended or overwrites existing routes, validation behavior, or reversibility. This is a moderate gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action, and contains no fluff. Every word adds relevant context about what the tool does and its flexibility.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the 100% schema coverage and absence of an output schema, the description covers the tool's purpose and scope effectively. It misses some behavioral context (e.g., route ordering, interaction with terminal flag) but is reasonably complete for a config-modifying tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by enumerating common handler types (reverse_proxy, file_server, etc.) and emphasizing 'full control', which helps the agent understand the flexible, free-form nature of the match and handle parameters beyond the schema's minimal examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Add a route') and its scope ('full control over match conditions and handlers'). It also lists specific handler types, which distinguishes it from sibling tools like caddy_remove_route and caddy_list_routes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'full control' implies this is the general-purpose tool for adding routes, and mentioning 'any Caddy handler' suggests flexibility. However, it does not explicitly state when to use this versus alternatives like caddy_reverse_proxy, nor does it provide any exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_stopA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false. The description adds meaningful context by specifying 'gracefully' shutdown and the confirm=true safeguard, which helps the agent understand the manner and safety requirement of the operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core action, and every word contributes value. There is no filler or redundant elaboration beyond the necessary confirmation note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter stop operation with clear annotations and full schema coverage, the description is sufficient. It does not explain return values or edge cases, but given the low complexity and existing structured data, that is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the confirm parameter with 100% coverage ('Must be true to confirm shutdown'). The description only repeats this requirement without adding new semantic detail, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'shut down' with a clear resource 'Caddy server', and it is distinct from sibling tools that handle config, status, or routes. The action and target are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use (to stop the server) but does not explicitly state when to use it versus alternatives or when not to use it. The confirm requirement is a prerequisite, not a usage guideline. It lacks explicit context for selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_reverse_proxyA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description thoroughly discloses behavioral traits beyond the annotations: it explains non-idempotent behavior when id is omitted, idempotent replacement when id is supplied, global @id collision refusal, and that the server argument is ignored once an @id is registered. This is far richer than the sparse annotations and significantly helps an agent predict runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately lengthy but every sentence serves a purpose: it states the core function, gives an example, explains idempotency conditions, warns about global ids, and notes a subtle server argument behavior. The structure is logical and front-loaded with the main purpose, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the lack of an output schema, the description covers the essential behavioral context: how to add a route, what happens with or without an id, and edge cases like id collisions. It does not describe return values or error responses, but for a config-mutation tool this is a minor omission when the operational behavior is so well explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers all parameters (100% coverage), but the description adds meaning by providing a concrete example for 'from' and 'to' and clarifying the nuanced behavior around 'id' (idempotency, global scope) and 'server' (ignored after initial registration). This goes beyond the schema descriptions, earning above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Add a reverse proxy route' with a concrete example, making the tool's function unambiguous. However, it does not explicitly distinguish itself from sibling tools like caddy_add_route, instead relying on the phrase 'The most common operation' to imply it is the standard route addition tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context that this is the 'most common operation' and shows a simple from/to example, implying it is for standard reverse proxy setups. It does not explicitly state when to use this tool versus alternatives such as caddy_add_route or when to prefer a different approach, leaving usage guidance implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_by_idA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false and provide little safety context, so the description carries the burden. It discloses that delete requires confirm=true, a key behavioral detail, and states the supported operations (read/update/delete). It doesn't elaborate on error handling or partial updates, but the confirm requirement adds meaningful transparency beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose, followed by scope and a critical safety note. Every sentence earns its place, and the formatting is clean and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 6 parameters and no output schema, the description covers the essential mental model: @id-based access, supported actions, and the confirm requirement. It doesn't need to enumerate every parameter since schema descriptions are thorough. A note about return values or error cases would help but isn't critical given the simplicity of the core operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal parameter-level meaning beyond the schema; it mentions the delete-confirm relationship which is already in the confirm field description. It does not elaborate on mode or subpath semantics, but the schema handles those adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool accesses config objects by @id, enabling read, update, or delete operations. It distinguishes itself from path-based alternatives by saying 'instead of needing its full path' and positions itself as the recommended approach for individual routes and config objects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this for individual routes/config objects with an @id, rather than full path. It implies when to choose this over sibling config tools, though it doesn't explicitly name alternatives or state exclusion criteria. The phrase 'recommended way' serves as a strong usage signal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_getA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds valuable behavioral context by explaining the return behavior for empty vs. specific paths, which goes beyond the structured annotations and helps the agent predict the tool's output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every word adds value. It avoids needless detail while including a practical example. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with one optional parameter and no output schema, the description covers the essential behaviors: full config vs. subtree, path example, and empty path default. It doesn't mention error handling or exact output format, but these are relatively minor gaps given the tool's simplicity and strong annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the only parameter 'path', but the description enriches its meaning by explicitly stating that an empty path returns the full config and providing a concrete example ('apps/http/servers/srv0/routes'). This goes beyond the schema's generic 'Config path' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads Caddy config at a JSON path, with a specific verb ('Read') and resource. It distinguishes itself from siblings by being the generic config getter, while other tools handle writes, deletes, or specific lookups. The clarification of empty path returning the full config adds scope precision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended usage is implied: use this to read config by path. However, there is no explicit guidance on when to prefer this tool over alternatives like caddy_config_by_id or caddy_status, nor any exclusions. The description provides clear context but no comparative direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_remove_routeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it destructive and idempotent. The description adds valuable behavior details: index-based removal is read-then-delete and can race against concurrent edits. This goes beyond the annotations and helps the agent understand risks.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action, and each sentence earns its place. The guidance is compact with no fluff, making it highly efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no required ones, and annotations cover safety. The description explains targeting and risks but does not mention return values or error behavior. Given no output schema, this is acceptable and sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all parameters, so the baseline is 3. The description reinforces the preference for @id and the race condition for index, but these are already largely present in the schema's parameter descriptions. The additional race warning is minor added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Remove a route' with a specific verb and resource, and distinguishes it from siblings like caddy_add_route and caddy_list_routes. It also outlines the two targeting methods, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use @id vs index, strongly preferring @id for stability and warning about race conditions with index. It does not compare directly with sibling tools, but the context is clear for when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_revertA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true, and the description adds valuable context: snapshots are kept in-memory (last 10), apply requires confirm=true as a safety, and list/save are non-destructive actions. This goes beyond the annotations, though it doesn't detail the exact behavior of apply (e.g., whether it restarts the server).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it opens with the tool's purpose, then explains the snapshot lifecycle and actions in a structured, easy-to-scan format. Every sentence adds necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has three distinct actions, and the description covers all of them with key details (timestamps, manual save, confirm requirement, retention limit). There is no output schema, but the description explains what each action produces. It could mention more about apply's effects, but the essentials are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters. The description adds some context by linking actions to parameters (e.g., 'apply' needs confirm=true, index for 'apply'), but this is minimal beyond the schema's existing descriptions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool manages config snapshots for rollback, with a specific verb ('Manage') and resource ('config snapshots'). It distinguishes from sibling tools like caddy_load by linking snapshots to auto-capture before caddy_load and enumerating specific actions (list, save, apply).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: snapshots are auto-captured before caddy_load, implying use this tool to inspect or undo changes from caddy_load. It also explains the actions and the confirm requirement for apply. While it doesn't explicitly list alternatives or exclusions, the context is sufficient to guide when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_upstreamsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds value beyond annotations by explicitly listing the output fields (address, active requests, failure counts), which is the primary behavioral detail for a status tool. It does not contradict annotations and provides useful context about what the agent will receive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately stating the action and resource, then enumerating the returned data. Every word is functional with no filler or repetition. It is front-loaded with 'Get the current health status' and remains compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema, strong annotations), the description covers the essential aspects: what it does and what it returns. It lists the three key fields, which is sufficient for an agent to understand the tool's output. However, it does not specify the exact response format or how health status is determined, leaving minor ambiguity for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty with 0 parameters, so the baseline is 4. The description correctly makes no mention of parameters, as none exist. It neither adds nor needs to add parameter semantics since there is nothing to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific function: getting current health status of reverse proxy upstreams. It lists the exact data shown (address, active requests, failure counts), distinguishing it from sibling tools like caddy_status (overall Caddy status) and caddy_reverse_proxy (likely configuration). The verb 'Get' plus the specific resource makes the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when you need upstream health information) but provides no explicit guidance on alternative tools or exclusions. It does not mention when not to use it or how it compares to caddy_status or caddy_metrics, leaving the agent to infer usage context from the purpose statement alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_adaptA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false. The description adds behavioral context: conversion is done locally ('without loading it'), returns 'adapted JSON and any warnings separately', and requires adapter modules to be compiled in. This exceeds annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the main verb 'Convert', with each sentence providing necessary detail (purpose, usage, output). No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description covers purpose, usage, prerequisites, and return format. It is self-contained for an agent to decide when to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions. The description enriches by listing built-in ('caddyfile') and external adapters ('nginx', 'yaml') and noting adapter module requirements, which adds meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'Convert a config in any registered adapter format to Caddy JSON without loading it.' This clearly identifies the action (convert), the resource (config), and the output (Caddy JSON). The 'without loading it' distinguishes it from loading-related siblings like caddy_load.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Useful for previewing what a Caddyfile produces, or for porting from nginx/yaml configs.' This gives explicit use cases. However, it does not explicitly name sibling alternatives or state when not to use, so it provides clear context but no exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_list_serversA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare this as a safe, read-only, idempotent operation. The description adds transparency about the output content (specific server attributes), which is helpful given there is no output schema. This goes beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the action and output details, followed by a practical usage hint. Every sentence contributes value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list operation with strong annotations, this description is complete. It tells the agent exactly what the tool does, what it returns, and when to invoke it, making tool selection and usage straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is fully covered by default. Per the baseline for 0-parameter tools, the description appropriately focuses on behavior and output rather than parameter explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the operation as listing all configured HTTP servers and specifies the exact data returned (names, listen addresses, route counts, TLS status). This distinguishes it from sibling tools like caddy_list_routes, which focus on routes rather than servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this to discover server names before calling route tools.' This provides a clear workflow context, though it does not explicitly name alternatives or exclusion cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe read-only nature is clear. The description adds valuable context about what the summary contains (servers, routes, listen addresses, TLS status), going beyond the annotations. It does not discuss failure modes, but the annotation coverage lowers the bar and the added context is meaningful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that packs significant information: the verb ('Check'), the resource ('Caddy connectivity'), and a detailed summary content list. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, no output schema), the description is complete. It explains both the action (connectivity check) and the expected return content (config summary with servers, routes, listen addresses, TLS status). Rich annotations further cover the safety profile, so the description sufficiently covers all necessary contextual information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so the schema is trivially complete (coverage 100%). Per guidelines, 0 params receives a baseline score of 4 because there are no parameter gaps to address. The description adds no parameter info, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Check Caddy connectivity and get a config summary'. It also enumerates the summary content (servers, routes, listen addresses, TLS status), which distinguishes it from sibling tools like caddy_config_get (full config) and caddy_list_servers (specific server list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating the tool provides connectivity checks and a config summary, but it does not explicitly recommend it over alternatives or mention when not to use it. It provides clear context for when this tool is appropriate, but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_deleteA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already mark this as destructive, but the description adds valuable context: deleting a parent node removes all descendants, and the confirm flag must be explicitly set to true. This goes beyond the basic destructive hint and clarifies real-world behavior, making it highly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core action. The first sentence states the purpose, followed by necessary cascade and safety details. There is a slight redundancy between 'Delete config' and 'Removes the config node,' but overall it is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, the description fully covers the destructive nature, cascading behavior, and the required confirm flag. With schema covering both parameters and annotations providing safety hints, no critical information is missing. The description is complete for safe and correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds extra meaning by explaining that deleting a path also deletes descendants, which affects how the 'path' parameter is interpreted. It reinforces the confirm requirement but doesn't add syntax details. This is a modest but meaningful addition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Delete') and resource ('config at a JSON path'). It distinguishes from siblings like caddy_config_set and caddy_config_get by explicitly focusing on deletion. The cascading behavior example further clarifies the scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool: 'Requires confirm=true' and explains the cascading deletion effect. While it doesn't explicitly mention when to prefer this over alternatives, the sibling context makes it obvious. No exclusions are given, but the guidance is sufficient for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_config_setA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes far beyond the sparse annotations by detailing HTTP semantics (PATCH/POST/PUT), idempotency per mode, and side effects like route duplication on repeated 'append' calls. It also highlights the 'insert' mode's utility for ordering. This fully discloses behavioral traits and potential pitfalls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (two sentences) and front-loaded with the core purpose before detailing modes. Each mode is explained with relevant consequences, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's behavior comprehensively: modes, idempotency, side effects, and ordering use-case. It doesn't mention return values, but the absence of an output schema makes that less critical. Given the complexity of the tool (three modes with different behaviors), this description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already well-documented. The description adds value by explaining the mode-specific behaviors (idempotency, side effects) that the schema enum descriptions only hint at. It does not add much for 'path' or 'value', but the schema already provides examples and definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Write config at a JSON path.' It specifies the resource (config), the action (write), and the path-based scope. This distinguishes it from sibling tools like caddy_config_get or caddy_config_delete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use each mode: 'overwrite' for safe/idempotent replacement, 'append' for adding to arrays (with warning about duplication), and 'insert' for route ordering. It doesn't explicitly mention alternatives to the tool itself, but the mode guidance is strong and context-rich.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_list_routesA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly/idempotent/non-destructive, and the description adds significant behavioral detail: output caps (500 routes/20000 chars), whole-route truncation, omission note, and exact fallback path. This fully discloses edge-case behavior without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences that each add distinct value: purpose/output, caps/truncation, and fallback. No redundant filler; structure is well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation with one optional parameter and no output schema, the description covers output format, limits, edge-case behavior, and how to get the complete data. It is fully self-contained for an agent to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'server' is already documented in the schema with default 'srv0' and 100% coverage. The description does not add additional parameter-level detail but doesn't need to given the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists all routes on a Caddy HTTP server and describes the dual output (summary + raw JSON). The use of 'all routes' and explicit reference to caddy_config_get for reading the rest distinguishes it from sibling config-read tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when you want a quick route overview on a specific server, and explicitly directs to caddy_config_get at a specific JSON path for the full untruncated data. The cap behavior and omission note also tell the agent when the result is incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_loadA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations already declaring destructiveHint and idempotentHint, the description adds significant behavioral detail: atomicity, a 60-second timeout for TLS provisioning, the mandatory confirm=true requirement, the fact that the entire running config is discarded, and that a snapshot is taken allowing restore via caddy_revert. These details greatly exceed what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, with the primary action front-loaded in the first sentence. Every sentence carries substantive information: format options, safety/timeout, destructive consequences, and snapshot/restore. There is no filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive, high-stakes operation, the description provides comprehensive guidance: it warns about the full config discard, mandates confirm=true, mentions atomicity and the timeout, and explains the snapshot/revert path. It also positions itself relative to large config changes. The absence of an output schema is not a meaningful gap since the return value is secondary to the behavioral warnings covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage by describing all three parameters, but the description adds complementary meaning: it clarifies that config can be a JSON object or a Caddyfile string, ties format='caddyfile' to that choice, and explains that confirm=true is required because the action discards the entire running config. This adds context beyond the schema, though it does not individually explain every parameter in depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Replace the entire Caddy configuration atomically,' using a specific verb ('replace') and a clear resource ('entire Caddy configuration') with a distinctive behavior ('atomically'). This distinguishes it from sibling tools like caddy_config_set or caddy_config_delete, which likely target partial changes. The mention of accepting JSON or Caddyfile further clarifies scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'This is the safest way to make large config changes,' providing clear context for when to use this tool. It also mentions restoration via caddy_revert, giving an alternative fallback. However, it does not explicitly say when not to use it (e.g., for small targeted changes) or name alternative tools like caddy_config_set, so it stops short of a full when/when-not distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• caddy_metricsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds substantial behavioral context: output can be megabytes, filter-mode drops blank/free-form comments but preserves HELP/TYPE lines, '# EOF' is always kept, max_lines defaults to 500, and a trailing comment reports dropped lines. This goes well beyond the structured hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized—four sentences that lead with the core purpose, then dive into parameter behavior with examples. Every sentence carries unique, essential information; there is no repetition or filler. The structure flows logically from purpose to usage to edge cases.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description covers the return format implicitly (Prometheus text) and explicitly covers filtering semantics, line handling, size limits, and the trailing summary. It gives the agent enough context to anticipate output characteristics and pitfalls, making it complete for invocation and result handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description significantly enriches parameter meaning. It explains that filter is a substring match on metric names, that HELP/TYPE lines for matching metrics are retained, and that label values are NOT matched. For max_lines, it details the default (500) and the appended summary line, which the schema does not fully convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Get Prometheus metrics from Caddy.' It lists concrete metric categories (request counts, durations, TLS handshake stats, active connections) and clearly distinguishes this tool from sibling tools like caddy_config_get or caddy_status, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use the filter and max_lines parameters, including examples and edge-case behavior (label values not matched). It does not explicitly contrast with alternatives, but among the siblings none offer metrics, so the context is clear enough for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.