Server Quality Checklist

Tool Scores

• coolify_application_managementC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it lists actions, it doesn't explain what each action actually does, their side effects, authentication requirements, rate limits, or error conditions. For example, it doesn't clarify whether 'deploy' overwrites existing deployments or what 'force' actually forces. This is inadequate for a multi-action mutation tool with 11 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured as a single sentence listing all available actions. It's appropriately sized for a multi-function tool, though it could be more front-loaded by grouping related actions (e.g., 'deploy and manage applications: create, list, get info, check status, start/stop/restart'). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 11 parameters, multiple actions (including mutations like deploy, create, stop), no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns, error conditions, prerequisites, or the relationships between actions and parameters. The agent would struggle to use this tool correctly without trial and error.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 11 parameters thoroughly with descriptions and enums. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain relationships between parameters (e.g., which parameters apply to which actions) or provide usage examples. The baseline of 3 is appropriate when the schema does all the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs comprehensive application management with specific verbs (deploy, get info, list, check status, start/stop/restart, create). It distinguishes itself from sibling tools by focusing specifically on applications rather than documentation, environment configuration, or system management. However, it doesn't explicitly differentiate between the various actions within the tool itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (coolify_documentation, coolify_environment_configuration, coolify_system_management). It also doesn't indicate when to use specific actions within the tool (e.g., when to use 'deploy' vs 'create', or when 'force' parameter is appropriate). The agent must infer usage from parameter descriptions alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• coolify_environment_configurationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'manage', 'configure', and 'retrieve' but doesn't clarify permissions needed, whether operations are destructive, rate limits, or what happens during failures. For a multi-action tool with potential mutations (update_env, set_domain), this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (8 words) and front-loaded with all key functionality. Every word earns its place by listing the three core capabilities without redundancy or fluff. It's appropriately sized for a multi-action tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 7 parameters, multiple actions (including mutations), no annotations, and no output schema, the description is insufficient. It doesn't address behavioral aspects, output format, error handling, or usage context. The agent lacks critical information to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema (e.g., it doesn't explain relationships between actions and parameters). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('manage', 'configure', 'retrieve') and resources ('environment variables', 'domains', 'application logs'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'coolify_application_management' or 'coolify_system_management', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, when to choose this over sibling tools, or any contextual constraints. The agent must infer usage solely from the action parameter enum.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• coolify_system_managementC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies read-only operations for listing and testing, but doesn't specify if these actions require authentication, have rate limits, or what the output format might be. For a system management tool, this lack of detail is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, using a single sentence with zero waste. It efficiently communicates the core functionality without unnecessary words, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of system management and the lack of annotations or output schema, the description is incomplete. It doesn't address behavioral aspects like authentication needs, error handling, or return values, which are crucial for an agent to use the tool effectively in a real-world context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single 'action' parameter with an enum. The description adds minimal value by listing the actions in a more readable format ('list servers, teams, services, or test connectivity'), but doesn't provide additional context or syntax details beyond what the schema already specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('list', 'test') and resources ('servers', 'teams', 'services', 'connectivity'), making it easy to understand what it does. However, it doesn't explicitly differentiate from sibling tools like 'coolify_application_management' or 'coolify_environment_configuration', which might have overlapping functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists possible actions but doesn't specify contexts, prerequisites, or exclusions, leaving the agent to infer usage from the action names alone without reference to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• coolify_documentationB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions actions like 'search docs' and 'get topic info' but doesn't describe behavioral traits such as rate limits, authentication needs, response formats, or error handling. For a tool with 4 parameters and no annotations, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: a single sentence that efficiently states the tool's purpose and key actions. Every word earns its place without redundancy or unnecessary elaboration, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no output schema, no annotations), the description is moderately complete. It covers the high-level purpose and actions but lacks details on behavioral traits, usage context, and output expectations. Without an output schema, it should ideally hint at return values, but it doesn't. It's adequate but has clear gaps for effective tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by listing actions (search, get, api, troubleshoot, topics, api_reference) and hinting at query usage, but it doesn't provide additional semantics beyond what's in the schema. With high schema coverage, the baseline is 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Access Coolify documentation' with specific actions like search, get topic info, API reference, and troubleshooting guides. It uses specific verbs (search, get) and identifies the resource (Coolify documentation). However, it doesn't explicitly differentiate from sibling tools like coolify_application_management, which likely handle different aspects of Coolify.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings (coolify_application_management, coolify_environment_configuration, coolify_system_management). It lists actions but doesn't specify contexts, prerequisites, or exclusions. Usage is implied through the action list but lacks explicit when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.