Server Quality Checklist

Tool Scores

• move_fileA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention overwrite behavior, whether directories are supported, or what happens on error. The only behavioral context is the sandbox-relative path requirement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short sentences, front-loaded with the primary action. Every word earns its place, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers the core function and path constraint, with an output schema present so return values need no explanation. However, with no annotations and negligible schema descriptions, it leaves unspecified behaviors like overwrite handling and directory support, which are relevant for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are empty (0% coverage), so the description must compensate. It adds that paths must be relative to the sandbox, which gives meaning to src_relative_path and dst_relative_path, but it does not explain formats or constraints beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Move or rename a file within the sandbox,' providing a specific verb and resource. It distinguishes itself from sibling tools like list_files, make_directory, and delete_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clarifies that both paths must be relative to the sandbox, which is a key usage constraint. However, it does not explicitly mention alternatives or when not to use the tool, though the purpose makes it obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral constraints: paths are relative to the sandbox root, '.' targets the root, and absolute paths are rejected. This adds valuable context beyond the schema. It does not describe error behavior or output specifics, but the existence of an output schema mitigates that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a single-sentence purpose followed by a bulleted list of three essential rules. Each rule earns its place, and the information is front-loaded with the main purpose. No unnecessary text is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description is complete enough. It covers the core functionality and key constraints. It does not mention error handling or hidden files, but these are not critical given the tool's simplicity and the presence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so effectively by explaining the 'relative_path' parameter: relative to sandbox root, '.' for root, and absolute paths not allowed. This add meaning beyond the raw schema definition, making the parameter's constraints and usage clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List files and folders inside the allowed sandbox directory.' It uses a specific verb and resource. However, it does not explicitly differentiate from the sibling tool 'list_root', though the mention of relative paths and the sandbox root provides some context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through the rules: paths must be relative, '.' lists the root, and absolute paths are disallowed. The description does not explicitly state when to use this tool versus alternatives like 'list_root', and no exclusions or prerequisites are provided. This provides clear context but lacks direct comparative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_rootA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It is a listing operation, which implies no side effects, but the description does not explicitly state that it does not modify anything or disclose any behavior like pagination or sorting. The added context about 'before organizing or cleaning up' is helpful but minimal, so a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, each earning its place. The first states the primary action, and the next two provide usage guidance. There is no fluff or redundancy, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with zero parameters and a simple list operation, the description covers the essential purpose and when to use it. An output schema exists, so return values are not required in the description. It is complete enough for the tool's simplicity, though it could mention that only immediate contents are listed, hence a 4.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the schema coverage is 100% and the baseline is 4. The description adds no parameter information because there are none; it correctly does not invent any. This matches the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List files and folders in the root of the sandbox,' giving a specific verb and resource. It distinguishes itself from sibling tools like list_files by explicitly scoping to the sandbox root, making it clear when this tool is appropriate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage scenarios: 'Use this to understand what files are available before organizing or cleaning up the directory' and 'Use this whenever the user asks what files are available or what files can be seen.' It gives clear context but does not explicitly mention alternatives when not to use it, hence a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_fileA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It clearly conveys the destructive nature (delete), scoping (sandbox), and a key limitation (files only). However, it does not disclose behavior on edge cases (e.g., file not found, directory passed, permissions) or whether deletion is permanent. These are meaningful gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with bullet points. The first sentence front-loads the core action, and the bullets add constraints without redundancy. Every word earns its place; no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no annotations), the description covers purpose, constraints, and parameter semantics adequately. It omits error handling and exact behavior on failures, but these may be covered by the output schema. Overall, it is sufficiently complete for a straightforward delete operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It fully explains the only parameter, relative_path, by stating it must be relative to the sandbox root. This adds essential meaning beyond the bare schema and eliminates ambiguity about path interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Delete') and resource ('file inside the sandbox'), clearly distinguishing it from siblings like make_directory (create) and move_file. It also specifies the scope (files only, not directories), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear contextual constraints: 'Only deletes files (not directories)' explicitly excludes directories, and 'Path must be relative to the sandbox root' sets a precondition. This gives the agent enough guidance on when and how to use the tool, though it does not explicitly reference alternatives or 'when not to use' beyond directories.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• make_directoryA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses two important behaviors: creates parent directories as needed and is idempotent if the directory exists. This goes beyond a bare statement and gives the agent actionable expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, using a brief lead sentence and bullet points. Every sentence delivers useful information without wordiness. The structure makes the key constraints and behavior easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no nested objects) and the presence of an output schema, the description covers the essential behavioral aspects. It leaves out error scenarios or permission requirements, but these are not necessary for a minimal sandbox directory creation tool to be used effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameter. It explicitly states that relative_path must be relative to the sandbox root, which adds meaning beyond the schema's bare type. Further detail on path format would be helpful but is not essential for this simple case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a directory inside the sandbox, using a specific verb and resource. It distinguishes itself from sibling tools like list_files, move_file, and delete_file, as none of those create directories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: paths must be relative to the sandbox root. It omits explicit when/when-not guidance compared to alternatives, but the distinct verb and resource make the intended use obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.