Server Quality Checklist

Tool Scores

• transform_dataA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explains the two operational modes and specific parameter behavior (omit field for flat arrays). However, it lacks disclosure of side effects, error handling, performance implications, or output format expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise—two sentences that front-load the general purpose and then dive into specifics. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and no annotations, the description adequately covers two major use cases but lacks details on edge cases, return structure (though an output schema exists), and behavior for all combinations of parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description adds critical meaning: it explains operation values, field omission rule, and output_format necessity. Yet input_format and data format expectations remain unspecified, leaving some parameters underdocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool transforms or aggregates tabular/record data, and specifies two modes: conversion and aggregation. It provides operation names and distinguishes between them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to use conversion (operation='convert') versus aggregation operations, and includes a special case for flat arrays. Although no comparisons to sibling tools are needed, it provides clear operation-specific context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_directoryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. The description discloses the base directory and relative path, but lacks details on permissions, error behavior for invalid paths, whether hidden files are listed, or depth of listing. Basic behavior is clear but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with front-loaded purpose. No redundant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Low complexity tool with one parameter. Output schema likely covers return format. The description provides the essential context (base path, relative paths) but could mention recursion or sorting behavior. Adequate for the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for the path parameter. The description adds meaning by explaining path is relative to /app and defaults to '.', which compensates. However, it does not mention if subdirectory traversal or patterns are supported.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists files and subdirectories within the sandboxed directory /app. The verb 'list' and resource are explicit. The sibling tools include read_file and write_file, so this is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for exploring the sandboxed directory. It explains path relative to root and default. However, it does not explicitly state when not to use or mention alternatives like read_file for file contents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and discloses key behaviors: reads UTF-8, respects sandbox boundaries, enforces size limits. However, it does not mention error responses or encoding handling beyond UTF-8.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with action, immediately clarify constraints. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 param, output schema exists), the description covers core constraints. It does not describe return format, but the output schema presumably does so.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to the 'path' parameter by clarifying it is relative to /app, which the schema lacks. However, it does not specify allowed formats or restrictions (e.g., no parent directory traversal).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads UTF-8 text files, specifies the sandboxed directory, and implicitly distinguishes from siblings like write_file and list_directory by focusing on reading file contents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context (sandbox, size limit) but does not explicitly state when to use this tool versus alternatives (e.g., list_directory for browsing, http_request for external files). Usage is implied but not contrasted.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_web_pageA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses key behaviors: it blocks non-public addresses, returns different content with/without selector, and specifies exact return components (title, text, links). However, it does not mention error handling, rate limiting, or behavior for invalid selectors, which are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with four sentences, each adding essential information: core function, selector behavior, security constraint, and return details. No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (so return format is covered), the description explains core behavior, selector usage, and security. However, it omits explanation of `max_links` and edge cases like non-matching selectors, leaving slight gaps for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains `url` (the page to fetch) and `selector` (CSS selector for targeted extraction), but does not explain `max_links` (default 50) – likely limiting outgoing links, but not stated. Partial compensation, one parameter unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches a public web page and extracts content, distinguishing behavior with or without a CSS selector. It also clarifies it blocks non-public addresses, making the purpose unambiguous and different from siblings like http_request or read_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for public web pages and mentions blocking of private addresses, but it does not explicitly guide when to use this tool over alternatives (e.g., http_request for raw content) or state when not to use it. Usage context is implied but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: UTF-8 encoding, sandbox directory, parent directory creation, and size limit refusal. Lacks details on overwrite=false behavior and encoding error handling, but good given no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with main action, no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers sandbox, encoding, directory creation, size limit, but lacks error handling details and default overwrite behavior. Output schema exists but not referenced.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Describes path as relative and content as UTF-8 text, but does not mention the overwrite parameter. With 0% schema coverage, partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it writes UTF-8 text to a file within a sandboxed directory, distinguishing from sibling tools like read_file and list_directory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage via sandbox context but does not explicitly state when not to use or provide alternatives beyond the sandbox restriction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• http_requestA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description discloses key behaviors: automatic credential attachment, ignoring Authorization headers, blocking non-public addresses, and redirect validation. No rate limits or error details are mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-load the purpose, then provide specific behavioral details and constraints. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers major behavioral aspects and constraints. Lack of parameter-level details is a gap, but output schema exists to describe return values. Could mention error handling or response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It adds context about methods and Authorization header restrictions, but does not explain parameters like body, json_body, query_params, or headers beyond the auth note. Parameter names are self-explanatory, but more detail would help.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: making authenticated HTTP requests to public APIs, listing supported HTTP methods, and explicitly distinguishing itself from sibling tools by noting it handles authentication and blocks non-public addresses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use (public APIs) and what not to do (avoid Authorization headers). However, it does not explicitly state when alternatives like scrape_web_page might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.