Server Quality Checklist

Tool Scores

• python_codeC
  2.6/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says code is executed and output is returned, but omits important behaviors such as sandboxing, filesystem access, network access, side effects, environment persistence, or how errors are surfaced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no real waste. 'This function will' is mild filler, but the core action and resource are front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a code-execution tool with no annotations and no output schema, this description is severely under-specified. An agent cannot know execution environment, side-effect risks, return value format, or constraints, which are critical for safely invoking a code runner.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It does clarify that the 'code' parameter contains Python source code, which is useful but largely redundant with the parameter name and tool title. It does not specify multiline formatting, execution context, or output structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('execute') and resource ('python code'), and notes that output is returned. It does not explicitly distinguish itself from sibling tools like python_file or bash, so an agent must infer that it runs inline Python code rather than a script file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. Sibling tools include bash and python_file, but the description does not explain that this tool is for inline Python code rather than shell commands or Python files.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does reveal the file-creation behavior, but it leaves the critical overwrite semantics unstated—would writing to an existing file replace it or append? It also omits permission errors and directory creation behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences, 18 words, and the main behavior is front-loaded. The opening 'This function will' is slightly padded, but overall the description is economical and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutating tool with no annotations and no output schema, the description should at minimum clarify whether an existing file is overwritten. It also does not specify parent-directory handling, path format, or failure modes, leaving an agent with meaningful uncertainty about side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only restates the parameter names ('file', 'content') in the phrase 'write the given content to a file' without adding detail on path formats, constraints, or edge cases. This does not meaningfully enrich the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('write') and resource ('file') and adds that a missing file will be created, so the core purpose is unmistakable. However, it does not explicitly distinguish this tool from sibling tools like python_file or bash, which can also write files, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use write_file versus alternatives such as bash, python_file, or read_file. The intended use is vaguely implied by the description, but no exclusions, prerequisites, or comparison to siblings are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bashC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only says that commands execute and output is returned. It hints at side effects through 'creating files, directories,' but does not disclose arbitrary filesystem mutation, destructive potential, shell environment, working directory, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the core behavior, but it repeats 'This function' and 'bash commands' across two sentences. The phrase 'and other bash commands' is redundant and adds little value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool, the basics of input and output are present. However, because annotations are absent and there is no output schema, the description should provide more context about side effects, execution environment, and error handling, especially for a potentially destructive tool like bash.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does add some meaning by explaining that 'command' is a bash command and giving examples, but it stops short of providing syntax, quoting, multiline command handling, or execution context details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action and resource: 'execute a bash command and return the output.' It also gives concrete examples like creating files and directories, which helps distinguish it from siblings such as read_file or python_code, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only usage guidance is that the function is 'useful for executing bash commands,' which is largely a restatement of its purpose. It does not explain when to prefer bash over siblings like write_file, create_folder, or python_code, and offers no exclusions, prerequisites, or safety conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• globC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the basic return behavior but omits important details such as whether paths are absolute or relative, whether matching is recursive, how hidden files are handled, and what happens when no files match.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short, front-loaded, and easy to parse. The second sentence adds context but is somewhat redundant with the first, keeping it from a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool without an output schema or annotations, the description is only minimally adequate. It gives the core behavior but leaves out pattern-language details and behavioral edge cases needed for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The pattern parameter has zero schema description coverage, and the tool description does not compensate by explaining expected glob syntax, examples, or supported wildcards. The agent is left to infer pattern semantics entirely from the parameter name and the tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the function returns a list of files matching a given pattern, which is a specific verb and resource. It does not explicitly differentiate itself from sibling tools, but the core purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says the function is 'useful for finding files in a directory,' which gives some context for when to use it. However, it does not mention alternatives, exclusions, or when a sibling like grep or read_file would be preferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• python_fileC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits but only states the basic execute-and-return-output behavior. It does not mention side effects, sandboxing, interpreter requirements, error handling, or whether the file is run in an isolated environment, which are important for a code execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler. It is front-loaded with the action and resource, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must provide essential context. It fails to cover return format details (stdout vs. full output), execution semantics, error behavior, or any safety-related context. For a code execution tool, this is a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explicitly explain what the 'file' parameter should contain. It implies a path to a Python file, but does not state whether relative paths are allowed, what file types are accepted, or any format constraints. The description adds minimal meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'execute a python file' and 'return the output.' This clearly communicates the core function and differentiates from file-read/write tools, though it does not explicitly contrast with the sibling tool python_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives like python_code or bash. There is no mention of prerequisites, working directory, or conditions that make this tool the appropriate choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grepB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does disclose the core behavior: it searches a file for a pattern and returns matches. However, it does not mention whether the pattern is a regex or literal string, or what happens on no match or a missing file, leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no wasted content beyond the filler phrase 'This function will'. It is appropriately concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter read-only search tool, the core input and output are stated. But with no output schema and no annotations, the lack of pattern-type semantics and no-match/error behavior makes it only minimally complete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to compensate. It only restates the parameter roles as 'given pattern' and 'specified file' without adding semantics like regex syntax, flags, or path handling. The description adds little beyond the parameter names already visible in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('search') with a clear resource ('specified file') and states the output ('return the matching lines'). It is clearly distinct from read_file (whole-file read) and glob (file discovery), though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to choose this tool over siblings such as read_file, glob, or bash. The usage context is only implied by the description, with no exclusions, prerequisites, or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the operation is a read and that the return value is a string, which implies non-mutation and gives the result type. However, it does not address error cases, file encoding, path resolution, or binary file behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler or redundancy. It leads with the action and specifies the return type immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 1-parameter read tool, the description covers the core purpose and output type, but it omits file-location semantics and error behavior. The lack of an output schema makes some return-detail gaps acceptable, yet the overall guidance is only minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only says 'a file' without explaining what the 'file' parameter should accept (path, filename, or handle), its format, or relative vs absolute paths. The parameter name alone is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('read'), the resource ('a file'), and the output ('return it as a string'). It distinguishes the tool from write_file but not from siblings like bash or python_file that could also read files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus bash, python_code, glob, or grep, and no mention of exclusions or alternative tools. The agent receives no help selecting among file-reading siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_folderB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does add one useful behavioral detail: the tool returns a message when the folder does not exist. However, it does not disclose whether deletion is recursive, permanent, or restricted to empty folders, which are important for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the primary action, followed by a relevant edge case. The phrase 'This function will' is mild filler, but both sentences contribute useful information and there is no unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema, but as a destructive operation with no annotations, the description should clarify permanence, recursion behavior, and whether the folder must be empty. It covers the non-existent case but leaves these important details unresolved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented 'folder' parameter. It only refers to 'the given name,' which adds minimal meaning beyond the schema's type string. No details about path format, absolute vs relative, or naming constraints are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific operation: deleting a folder by name. It is unambiguous and naturally distinguishes itself from sibling tools like create_folder, read_file, and write_file, so an agent can identify the correct resource and action without additional context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives such as bash, which could also delete folders. There is no mention of prerequisites, constraints, or scenarios where another tool would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_folderB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does mention the duplicate-folder behavior, which is useful, but it omits other behavioral traits such as whether relative paths are supported, permissions required, side effects, or what the success return looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and both contribute useful information. The only slightly wasteful phrase is 'This function will', but overall it is efficient and front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool, the description covers the core operation and the duplicate case. However, it does not describe the success response, the structure of the duplicate message, or any path-related behavior, and there is no output schema to fill those gaps. Adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented 'folder' parameter. It does clarify that the folder parameter is the name, but it fails to specify whether it should be a bare folder name, a path, or any formatting constraints. Minimal compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb and resource: create a folder with a given name. Among sibling tools, only delete_folder relates to folders, and this describes the inverse operation, so there is no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to choose this tool over alternatives like bash or write_file, nor are any prerequisites or conditions stated. The description only explains what the tool does, not when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.