Server Quality Checklist

Tool Scores

• deploy_docker_composeA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It clearly states the tool clones/updates a git repo, installs Docker if missing, writes a .env file if provided, and runs docker compose up -d --build. This covers major side effects (Docker installation, file creation). However, it does not mention prerequisites like git existing, potential overwriting of existing files, or what happens on failure. Still, it is fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the main action and lists key steps. Every phrase is necessary, and there is no fluff. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, no output schema, and no annotations, the description covers the main workflow but misses important details like parameter relationships, success criteria, or what happens when Docker is already installed. The sibling tools are diverse, and more context on when to use this over exec or write_file would improve completeness. It is adequate but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all 9 parameters with individual descriptions. The tool description adds minimal extra meaning beyond providing a high-level workflow (e.g., 'writes a .env file if provided' maps to envFileContent). With full schema coverage, a baseline of 3 is appropriate, and the description does not significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title and description clearly state it deploys a docker-compose project. The description enumerates specific steps (clone/update git repo, install Docker, write .env file, run docker compose up -d --build), making the purpose very specific. It distinguishes from sibling tools like deploy_python_bot, exec, and write_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for docker-compose projects but provides no explicit guidelines on when to use or avoid this tool. No alternatives are mentioned, and there is no discussion of prerequisites or scenarios where exec or write_file might be preferred. The context from sibling tools offers some implicit differentiation, but the description itself lacks guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations so description must carry full burden. States it writes via SFTP, but lacks details on overwrite behavior, error handling, or permission requirements. Adequate for basic understanding but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the core action, no redundant information. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters, 5 required, and no output schema or annotations, the description could include more about return behavior, overwrite safety, or prerequisites. Still, it covers the primary purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all parameters described. Description adds no extra parameter information beyond the schema, so baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the action (writes text content via SFTP), the resource (file path on remote server), and distinguishes from sibling deploy_* tools by specifying use cases like config files and scripts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit context when to use (config files, env files, scripts) and hints at alternatives (deploy_* recipes). Missing guidance on when not to use relative to exec or other tools, but sufficient for most agents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• setup_nginx_sslA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool installs software and modifies configuration, which implies system-level changes. However, it does not explicitly state that existing nginx configurations may be overwritten or that the tool requires root privileges.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and every word adds value. No redundant or verbose phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a setup tool with 8 parameters and no output schema, the description covers the essential workflow and prerequisites. It is sufficient for an agent to determine when to invoke this tool and what it accomplishes, though details on post-execution state are omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already explains each parameter well. The description adds minimal extra context beyond the prerequisites. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool installs nginx and certbot, configures a reverse proxy, and obtains a Let's Encrypt SSL certificate. These specific actions distinguish it from sibling tools like deploy_docker_compose or exec, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly specifies prerequisites: DNS A record must point at the server and ports 80/443 must be open. This tells the agent when the tool can be used, though it does not directly mention when not to use it or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deploy_python_botA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description outlines key behaviors (clones/updates repo, creates venv, installs requirements, writes systemd unit, starts service). However, it lacks details on idempotency, error handling, SSH key management, and what happens if the service or directory already exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first covers the core actions, second provides purpose and exclusion. No redundant information; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters, no output schema, and no annotations, the description covers main behaviors but misses important details like idempotency, SSH key handling, and handling of existing deployments. Adequate but has gaps that could confuse an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds minimal meaning beyond attribute names. It references entryPoint and appName in context but does not elaborate on parameter formats, constraints, or relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool clones/updates a git repo, creates a venv, installs requirements, writes a systemd unit with auto-restart, and starts it. It specifies it is for long-running Python processes like aiogram Telegram bots and distinguishes from sibling tool setup_nginx_ssl by stating what it does NOT do.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance: designed for long-running Python processes (e.g., Telegram bots) and explicitly states when not to use (when a reverse proxy is needed) with a direct alternative: 'use setup_nginx_ssl separately'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• view_audit_log_pathA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the tool is read-only and returns a file path with each command logged as one JSON line. No side effects or hidden traits are needed for this simple operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core function, and no wasted words. The first sentence states the action and result, the second gives usage advice. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and a straightforward purpose, the description covers everything needed: what it returns and what to do with it. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% (vacuous). As per guidelines, a baseline of 4 is appropriate since the description does not need to add parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a local file path for audit logs, specifying the exact resource (audit log file) and the action (returns path). It distinguishes from sibling tools which are all about deployment and file writing, none providing audit log paths.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to tell the user this path if they want to review history, providing clear context for when to use. It does not discuss alternatives or when not to use, but given the tool's simplicity, this is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it runs a command remotely, returns stdout/stderr/exit code, and blocks dangerous commands unless confirmed=true. It also instructs how to handle blocked commands by explaining risks and getting explicit user approval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three efficient sentences with no wasted words. The description is front-loaded with the core action and return format, then usage guidance, then safety protocol.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explicitly mentions the return values (stdout/stderr/exit code). It covers when to use, dangerous command handling, and basic parameters. For a complex tool with 7 parameters, it provides sufficient context without being overly verbose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the confirmed parameter in context of dangerous commands and implying the use of host and username. It gives additional semantic meaning beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a single shell command over SSH and returns stdout/stderr/exit code. It distinguishes itself from sibling deploy_* recipe tools by explicitly stating to use it for anything that doesn't fit those recipes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: 'Use this for anything that doesn't fit one of the deploy_* recipe tools.' It also provides guidance on dangerous commands, explaining the blocking mechanism and how to handle it with user confirmation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.