Server Quality Checklist

Tool Scores

• run_powershellC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full behavioral disclosure burden. It only says the command is 'restricted' but does not explain what restrictions apply, whether execution is sandboxed, whether side effects are possible, or how output is returned. For a command execution tool this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence with no filler. It front-loads the operation and adds useful qualifiers like 'restricted' and 'development command' while remaining compact, though it is too sparse to be fully informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool executes arbitrary commands, which is high-risk and context-dependent, yet the description provides no details about restrictions, expected use cases, or behavioral limitations. Even with an output schema present, the lack of any guidance around command selection, permissions, or failure modes makes this incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain any of the three parameters. The agent is left to infer the meaning of command, timeout_seconds, and working_directory solely from their names and types. The description adds no parameter-level meaning beyond what the schema already exposes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a PowerShell command, identifies the platform as Windows, and narrows scope to 'development' commands. It distinguishes itself from sibling file, build, and network tools, though 'restricted' is somewhat vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given about when to use this tool versus alternatives such as build_project, compile_shader, or get_system_info. The phrase 'development command' weakly implies a context, but there are no conditions, exclusions, or alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_system_infoB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. It only states that information is returned, without noting that the call is read-only, whether it can fail, require permissions, or have side effects. For a no-annotation tool, more behavioral context is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. Every word contributes to stating the tool's purpose, making it appropriately sized for a zero-parameter getter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-argument tool with an output schema, the basic purpose is covered and return value details are not required. However, it lacks usage guidance relative to siblings and provides no behavioral guarantees, leaving the description minimally viable rather than complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is complete, so there is no parameter meaning that the description must clarify. This aligns with the baseline of 4 for parameter-less tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Return information about the system this server is running on.' It clearly identifies the tool as a system-information getter, and the phrasing 'this server is running on' distinguishes it from server-level siblings like get_server_info, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as get_server_info or get_gpu_info. The description implies it should be used when system information is needed, but it provides no exclusions, prerequisites, or context-dependent selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_fileB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description transparently discloses that files always land in the download root, arbitrary destinations are impossible, existing files are never overwritten, and the 500MB maximum is enforced. This is far beyond a generic 'downloads a file' statement, though it omits auth/error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs with the core action in the first sentence. Every sentence adds a distinct safety or behavioral fact, with no padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the critical constraints: domain approval, destination root, no overwrites, size limit, and the output schema fills in return structure. It lacks explicit parameter semantics and usage routing, but remains largely sufficient for safe invocation of a 2-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description never explicitly explains the url or filename parameters. It indirectly implies behavior (filename is a suggested name within the download root) but leaves agents to infer how filename maps to the final path and whether url must be HTTPS beyond the initial sentence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The phrase 'Download a file from an approved HTTPS domain' gives a specific action, resource, and key constraint. It clearly distinguishes the tool from content-fetching siblings by emphasizing saving to disk, though it never names alternative tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No statement of when to choose this over sibling tools such as fetch_web_page or fetch_https_response. The only hint is the word 'Download' versus content retrieval. No exclusions or conditional usage guidance are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_github_latest_releaseA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the burden. It discloses an external call to the approved api.github.com domain and a return of metadata, but it does not mention rate limits, auth, or failure modes. For a read-only metadata fetch, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, main purpose first, no filler. The example and domain note each add practical value without bloating the description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return structure is already specified. For a single-parameter external read tool, the example plus domain note cover most needs; the main omission is explicit when-to-use guidance, which is scored separately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides only a bare 'repository' string with 0% coverage. The example 'premake/premake-core' adds the owner/repo format implicitly, which is useful, but the description never explicitly explains the required format or any constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Return the latest GitHub release metadata'), names the domain, and gives a concrete example. This is immediately distinguishable from generic web/file tools in the sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when/when-not guidance or alternative tool is named. It only implies GitHub-specific use via the example and domain, but does not tell an agent when to prefer this over fetch_web_page or fetch_https_response.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_directoryA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the operation is read-only and explains key behaviors: plain-text output, sorting order, and truncation of large directories instead of refusal. This goes well beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose. Every sentence adds useful information: output format, ordering, truncation behavior, and read-only safety. There is no wasted wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read-only tool, the description covers the main operational behaviors: sorting, truncation, and non-destructiveness. It does not address error cases or path edge cases, but the presence of an output schema reduces the need to document return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to compensate by explaining the path parameter's expected format or semantics. It does not mention how paths should be specified, whether relative/absolute paths are accepted, or how the default '.' behaves. The single parameter is somewhat self-evident from its name, but the description adds no parameter-level meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action and resource: listing the contents of a directory. It adds meaningful detail—directories first, then files, alphabetically—which distinguishes it from sibling tools like read_file, find_files, and search_files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied: use this when you want a plain listing of a directory's contents. However, it does not explicitly mention alternatives or when not to use it, such as when needing recursive file discovery via find_files or empty-directory detection via list_empty_dirs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_empty_dirsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It explains the recursive emptiness condition and the top-down reporting order, which is valuable. It does not mention error handling or permissions, but the tool's non-destructive nature is clear from the verb 'list'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence summary followed by a concise explanation of the recursive definition and its practical value. Every sentence adds necessary information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return value documentation is not required. The description covers the core behavior and rationale. It omits edge cases like hidden files, symlinks, and permission errors, but for a 1-parameter list tool with an output schema, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It indicates 'path' is the starting point ('beneath a path') but does not specify whether it must be a directory, absolute vs relative, or behavior for non-existent paths. The single parameter is simple enough that this is adequate but not rich.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Find empty directories beneath a path.' It also precisely defines 'empty' recursively, clearly distinguishing this from sibling tools like list_directory or find_files. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a cleanup use case ('which is what makes the result usable for cleanup') but does not explicitly state when to prefer this tool over alternatives like find_files or list_directory, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes beyond a simple summary by explaining the raw text return format, the rationale against JSON wrapping, truncation policy for long files, and how continuation after truncation is reported. It also explicitly states the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose. Every sentence earns its place: raw output format, truncation behavior, and read-only status. It is neither padded nor under-specified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple file-reading tool with an output schema present, the description covers the most important behavioral concerns: raw text, truncation, continuation, and read-only safety. However, it omits parameter-level semantics and explicit routing among siblings, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for explaining path, max_lines, and start_line. It does not describe any of these parameters explicitly, and while it mentions truncation, it never ties that behavior to the max_lines parameter or explains how start_line and max_lines interact. Parameter names and defaults exist in the schema, but the description adds no semantic guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Read a UTF-8 text file.' It clearly distinguishes this from siblings like write_file, edit_file, list_directory, and find_files by establishing the read-only nature and the target content type. There is no ambiguity about what the tool accomplishes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this is the tool for reading text files and is explicitly read-only, which implies it should not be used for writing or editing. However, it does not explicitly name alternatives or provide when-not-to-use guidance, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filesA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden and does so well. It discloses recursive traversal, line-level result reporting, automatic skipping of build/version-control directories and binary files, and why that matters for performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: short lead sentence, concrete use cases, then the key skipping behavior and rationale. Every sentence earns its place and no space is wasted on generic filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately simple search tool, the combination of description, schema defaults, and output schema is largely sufficient. The main gap is the lack of parameter-level semantics, but the parameter names and defaults make the common invocation pattern clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate for it. It implies that pattern is searched in file contents, but it does not explain regex, file_glob, ignore_case, max_results, or path behavior beyond what the bare parameter names and defaults suggest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: recursively searching file contents and reporting matching lines. The examples 'where is X used' and 'what creates the swapchain' make it clear this is content search, distinguishing it from filename-based search and from reading files individually.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to prefer this tool over reading files one by one and frames it as the cheapest way to locate code in a large tree. It does not explicitly point to find_files as the filename-search alternative, so the exclusion guidance is not quite complete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compile_shaderA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and handles it well: it discloses the output file location, write confinement to download root or BIONIC_PROJECT_ROOTS, automatic glslangValidator fallback, and error reporting with file and line. It could mention overwrite behavior, but this is strong disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is front-loaded, and the extension-to-compiler rules are organized in a scannable bulleted list. Every sentence conveys necessary information without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers compiler selection, required flags, fallback behavior, output location, and write confinement. An output schema exists, so return values do not need elaboration. The only significant gap is the undocumented `working_directory` parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains source extension behavior, `stage`, `profile`, `spirv`, and `output`. However, `working_directory` is never mentioned or explained, leaving one of six parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Compile a shader and report errors with file and line.' The extension-to-compiler mapping makes it unmistakably a shader compilation tool and separates it from sibling file and project tools like build_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear parameter-level usage conditions: .glsl requires `stage`, .hlsl and .fx require `profile`, and `spirv=true` changes the target. It does not explicitly say when to prefer this tool over build_project, so it stops short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_https_responseA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the allowed-domain restriction, the 2 MB response cap, and the text-only nature of the response. It does not discuss error behavior or authentication, but for a simple HTTPS text fetcher the disclosed constraints are meaningful and sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core action. The bullet list of use cases is scannable and every sentence contributes useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema, so the description does not need to detail return values. It covers the main constraints, use cases, and limitations. It could slightly improve by naming sibling alternatives, but overall it is complete enough for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and only one parameter, 'url'. The description compensates by clarifying that the URL must be HTTPS, belong to an approved domain set, and yield a text response no larger than 2 MB. This adds real meaning beyond the bare 'url' field, though it does not provide an exhaustive allowed-domain list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific action and resource: 'Fetch text from an approved HTTPS URL.' It clearly distinguishes itself from siblings like download_file (binary content) and read_file (local files) by emphasizing text-only output from approved remote HTTPS sources. The 'Useful for' list reinforces its intended scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: fetching release metadata, documentation, GitHub API responses, and text configuration files. It also states the domain restriction, which implicitly tells the agent this is not for arbitrary web pages. However, it does not explicitly contrast with fetch_web_page or download_file.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_filesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses recursion and the automatic skipping of build/vcs directories, which are not inferable from the schema. It could add nuance about result limits or matching rules, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs, each earning its place: the first line defines the action, the second gives usage context, and the third warns about skipped directories. The key facts are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with an output schema, the description covers purpose, scope, exclusions, and an alternative tool. It is missing explicit parameter semantics for max_results, but the existence of an output schema reduces the need to document return shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies file_glob as a name pattern and path as a recursive tree search, but it does not explain glob syntax or the max_results parameter's truncation behavior. Adequate but with clear gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a clear verb and resource: 'List files matching a name pattern, recursively.' It explicitly contrasts with list_directory ('shows one level; this searches the tree'), making the tool distinguishable from a key sibling at a glance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States when to use it: 'Use this to answer what exists before reading anything,' with concrete examples. It also names an alternative and the distinction (list_directory for one level, this for tree-wide search), satisfying the when-to-use vs alternative requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it adequately conveys that this is a read-only informational call by saying 'Return information'. It goes beyond a simple metadata listing by explaining the diagnostic nuance that it can distinguish disabled vs missing web research tools, which is behavioral context not available in the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loads the core purpose, and each sentence adds value. The second sentence expands the return contents and adds a distinctive diagnostic detail without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, an output schema, and a clear description of the return scope, nothing important is missing. The description even provides the most valuable practical context: using it to detect disabled vs missing web research tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter semantics to document. The baseline for zero-param tools is 4, and the description appropriately focuses on return value rather than input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Return information about this MCP server', and immediately distinguishes it from sibling tools like get_system_info and get_gpu_info by clarifying the subject is the MCP server itself. It further specifies meaningful contents (version, transport protocol, configuration), so there is no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear diagnostic use case: checking whether web research tools are available, and notes this is the only way to distinguish a disabled tool from a missing one. It doesn't explicitly name alternatives, but with sibling tools being unrelated (git, files, system hardware), no exclusion is really needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_web_pageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and excels: it warns the page is untrusted and may contain agent-directed instructions, forbids running suggested commands, explains config-dependent host readability (default allowlist vs research mode), prescribes refusal etiquette (report once, never retry a different URL on the same host), discloses short-lived caching, and pins protocol constraints (HTTPS-only, port 443, no credentials, private addresses). There are no annotations to contradict.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long, but every paragraph earns its place: core purpose, usage context, security warning, host configuration (cleanly bulleted), refusal handling, and protocol constraints. Dense without redundancy and front-loaded with the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a security- and configuration-sensitive network tool with an output schema and zero annotations, the description fully covers purpose, triggers, safety behavior, host policy, failure handling, and constraints. Return formats are already documented by the output schema, so nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does add real meaning: URL constraints (HTTPS-only, port 443, no-credentials, private-address rules) and the paging/caching behavior via the read_file convention. It stops short of spelling out how max_lines and start_line interact or what the defaults imply, relying on cross-reference to read_file.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb+resource: 'Read a web page as plain text.' The 'plain text' qualifier meaningfully distinguishes it from the closest sibling fetch_https_response (raw HTTP fetch) and from read_file (local files), and 'Use this after web_search' positions its role clearly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Use this after web_search to read a result' gives explicit when-to-use context, and the paging-convention note ties it to a known workflow. However, it never states when not to use it — e.g., when fetch_https_response would be appropriate instead — so no exclusion or alternative-selection guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_gpu_infoA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers extensively. It discloses the read-only nature of the probes, the use of WMI and vulkaninfo, per-probe time limits, and graceful degradation when a driver is broken. It also states an important behavioral boundary: Direct3D feature levels are deliberately not reported and would be a guess if included. This is far beyond the minimum required.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then expands with concrete detail that earns its place: the exact questions answered, the failure mode it prevents, the probing mechanism, and a crucial exclusion note. Every sentence adds decision-relevant information without padding. It is longer than average, but the complexity of the tool's scope justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters, no annotations, but does have an output schema, the description covers everything an agent needs to invoke it correctly: what it reports, how it gathers data, how it behaves under failure, and explicitly what it does not report. The presence of an output schema means return-value details are already provided elsewhere, so nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is an empty object with 100% coverage by definition. The baseline for zero parameters is 4 because there is nothing for the description to add about individual argument semantics. The description instead clarifies what information is gathered, which is appropriate for this parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description leads with a specific verb and resource: 'Report the GPU adapters, driver versions and graphics toolchain.' It then enumerates the exact questions it answers (adapter, driver, Vulkan SDK, shader compilers, Vulkan API version), which clearly differentiates it from generic siblings like get_system_info. There is no ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: a renderer needs this information before generating code, and missing it causes guesses about extension and feature support. It also explicitly notes what is NOT covered (Direct3D feature levels) and directs the user to query that from the application. It does not name a specific sibling alternative, but the scope is so well defined that an agent can route correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_projectA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and it delivers. It discloses output parsing behavior (unique errors first with file/line/code, repeat counts), compiler support (MSVC, clang, gcc, CMake, ninja), and fallback behavior when nothing parses (tail of raw output instead of misleading 'no errors'). It also warns about restrictions on command execution. This is substantial behavioral context beyond the name and schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is multi-paragraph but every sentence earns its place: core purpose first, then rationale for preferring it, then parsing details, then constraints. No fluff. The structure moves from high-level intent to operational specifics without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity—a build tool with parsing logic and restrictions—the description covers the essentials: what it does, when to use it, what commands are allowed, what output to expect, and failure fallback. The presence of an output schema further reduces the need to describe return types. Nothing critical is missing for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It does explain the 'command' parameter thoroughly—listing allowed drivers and constraints. However, it does not address 'timeout_seconds' or 'working_directory'. Their names are self-explanatory and defaults are in the schema, but the description adds no additional detail about their behavior or defaults. Still, the most complex parameter is deeply documented, so it earns a strong score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run a build command and return only the diagnostics that matter.' It clearly distinguishes itself from run_powershell by explaining the parsing/deduplication behavior, and the scope (build drivers) is explicit. An agent can immediately tell what this tool does and how it differs from its closest sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'Prefer this over run_powershell for anything that compiles' and explains why. It also enumerates the allowed build drivers and states constraints (exactly one command, no pipelines/redirection/separators outside quotes). This gives the agent clear decision rules and boundary conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden, and it delivers: it explains overwrite refusal, the destructive risk of full rewrites, filesystem confinement to download root and BIONIC_PROJECT_ROOTS, automatic creation of missing parent directories, no CRLF translation, byte-for-byte newline fidelity, and UTF-8 encoding. This is exemplary disclosure for a mutating file tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized and front-loaded: the purpose appears first, followed by the critical overwrite caveat, sibling routing, filesystem constraints, and encoding guarantees. Every sentence adds operational value; nothing is filler or redundant with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutating tool with no annotations and no schema-level parameter descriptions, this description is remarkably complete. It covers success behavior, failure/refusal behavior, scope restrictions, edge cases like missing directories, and byte-level output guarantees. The presence of an output schema means return value details are not required in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all parameters. It explains overwrite semantics clearly, path confinement, parent-directory creation, and content encoding/newline behavior. The only minor gap is that the exact path format or how roots are specified is not directly stated, but the confinement rule gives enough practical guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Create a text file, or replace one entirely.' It clearly distinguishes itself from sibling edit_file by stating that partial changes should use edit_file and that write_file is reserved for creating new files (or full replacements). An agent can tell exactly what this tool does and what it is not for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: use write_file for creating new files, use edit_file for changing part of a file. It also explains the deliberate default of refusing overwrite unless overwrite is true, and warns against using full-file rewrites for small changes. This is strong routing and usage instruction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_fileA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure, and it does so thoroughly. It explains exact-match and uniqueness requirements, the refusal behavior when a match is ambiguous, automatic line-ending detection and preservation, and the filesystem confinement rules. These are meaningful behavioral traits beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long; but each sentence adds distinct value: purpose, authoring guidance, matching rules, line-ending behavior, and scope constraint. It is front-loaded with the core purpose and the most important usage discrimination, making it easy for an agent to process.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This tool is a mutation with four parameters and no annotations, yet the description provides all essential context: what it does, when to choose it, how matching works, what happens in ambiguous cases, line-ending behavior, and filesystem boundaries. An output schema exists, so lack of return-value details is not a gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage, so the free-text description is the only semantic source. It compensates well: old_string is documented with exact matching, uniqueness, and ambiguity handling; replace_all is explained as the disambiguation alternative CUERPOS paramContext; new_string is implicitly covered through line-ending handling and replacement semantics; path is constrained by the download-root/BIONIC_PROJECT_ROOTS statement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-plus-resource statement: 'Replace an exact string in an existing text file.' It also explicitly differentiates itself from the sibling write_file by stating it is 'Preferred over write_file for changing existing code' and explains why, so an agent can distinguish the tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to prefer this tool over write_file and gives the rationale (only the named region moves, so the rest of the file cannot be lost). It also provides actionable guidance for problematic cases: include surrounding lines to disambiguate duplicates or pass replace_all.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.