Server Quality Checklist

Tool Scores

• cacheC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only mentions TTL and namespaces, but fails to describe important behaviors such as how TTL expiration works, whether delete and clear are destructive and irreversible, whether clear operates per namespace or globally, or what happens to expired keys. These are significant gaps for a tool with mutation operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that establishes the core resource and lists all operations without any filler. It is front-loaded and every element contributes to understanding the tool's scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-operation tool with no output schema, the description is too minimal. It does not explain what each operation returns (e.g., get returns a value, list returns keys), nor does it clarify important side effects like the destructive nature of delete/clear or TTL enforcement with the default namespace. This undercuts the agent's ability to predict behavior correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% parameter description coverage, including descriptions for operation, key, value, namespace, and ttl_seconds. The description adds no additional parameter semantics; it merely repeats TTL and namespaces and lists operations that are already defined in the enum. Baseline of 3 is appropriate because the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource as a key-value cache with TTL and namespaces, and enumerates the supported operations (set, get, delete, list, clear). It is specific enough to distinguish from sibling tools like database-query or file_read, though it lacks a single verb+resource construction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention that it is for temporary data, that database-query might be preferred for persistent structured data, or any exclusions. There is no 'when to use' or 'prefer X instead' statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_createB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions key behaviors: tasks are asynchronous, have a TTL, and can optionally trigger webhook notifications on status changes. However, it does not disclose authentication requirements, return values, or side effects beyond the TTL and webhook behavior. Some transparency is provided, but significant gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences that front-load the primary purpose and then summarize key features. Every sentence contributes useful information without redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool creates a task but there is no output schema, so the description should explain what the response contains (e.g., task ID, status URL). It also lacks guidance on permissions, error conditions, or lifecycle behavior beyond the TTL. For a creation tool with moderate parameter complexity, the description is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all five parameters. The description adds marginal value by highlighting TTL and webhook features, which map to existing parameters, but it does not provide additional semantic context beyond what the schema already offers. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create an async task for tracking long-running operations.' It uses a specific verb and resource, making the core action unambiguous. It does not explicitly differentiate from sibling tools like task_status, but the wording is specific enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage context: 'for tracking long-running operations,' indicating when the tool is appropriate. However, it lacks explicit guidance on when not to use it or how it compares to sibling tools like task_status or webhook_register. The guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_searchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that results are 'relative paths within the sandbox', which is a key behavioral trait. However, it omits other relevant behaviors such as recursion semantics (implicitly via glob), hidden file handling, or error/empty results behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences, each earning its place. The first states the purpose and gives an example; the second clarifies the output format and scope. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter and no output schema, the description is largely complete for a simple search tool. It states what it returns and the sandbox scope. Minor gaps exist (e.g., no statement about case sensitivity or hidden files), but these are not critical for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the pattern parameter with the same example ('**/*.ts'). The description adds no information beyond the schema, so the baseline score of 3 is appropriate. The example in the description is redundant with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search for files matching a glob pattern' with a concrete example. This specific verb+resource combination distinguishes it from sibling tools like file_list or file_read, and the 'Returns relative paths within the sandbox' scopes the operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It does not mention file_list or other siblings, nor does it describe criteria for choosing search over listing or reading. The example pattern is helpful but not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_readA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the file type and path constraint but does not disclose behavior on missing files, return format, or whether the operation is strictly read-only. For a read tool, this is minimal but expected; however, it lacks explicit non-destructive assurance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler. It front-loads the core action and adds the critical path constraint efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple file read tool with two parameters and no output schema, the description covers the basics but omits details like return value semantics and potential errors. It is adequate for typical use but not exhaustive; more context could be given on encoding behavior or binary file handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description adds meaningful clarification that the path is relative to the sandbox base directory, which is a constraint not fully captured in the schema's brief description. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads the contents of a text file, with a specific verb and resource. It distinguishes itself from sibling tools like file_write, file_list, and file_info through the name and the explicit 'read' action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a key usage constraint—path must be relative to the sandbox base directory—but offers no explicit guidance on when to use this tool versus alternatives like file_info or file_search. Usage context is largely implied by the tool's name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• webhook_registerA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the optional HMAC secret for signature verification, but it does not disclose side effects, permissions, reversibility, or what response to expect. This is a significant gap for a mutation tool that registers a webhook.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the primary purpose and then lists key inputs. Every clause earns its place, with no redundant or fluffy wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and no output schema, the description provides a basic understanding but lacks detail on return values, registration persistence, or how to use the secret in practice. It is adequate but leaves gaps that the schema does not fully fill (e.g., what happens after registration).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with descriptions, so the baseline is 3. The description adds no extra meaning beyond echoing parameter names (URL, event types, secret). It does not explain parameter formats or relationships beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Register a webhook callback URL to receive event notifications.' It uses a specific verb ('Register') and resource ('webhook callback URL'), and the purpose is explicit. It distinguishes from sibling tools like webhook_send by focusing on receiving notifications via a registered URL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: 'to receive event notifications.' It implies use cases like subscribing to events and setting up a callback. However, it does not explicitly mention alternatives or when not to use it, which prevents a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• api-connectorA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the tool makes HTTP requests, returns status/headers/body, and supports retries and configurable timeouts. However, it does not specify error handling, redirect behavior, response size limits, or security implications, which are significant for a generic HTTP client.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and includes only relevant details about features and return values. Every clause conveys necessary information without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters and no output schema, the description adequately explains the return value as status, headers, and body, and mentions the core features. However, it does not specify the response object structure or error behaviors, leaving some context gaps that the schema does not cover.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with detailed descriptions, so the baseline is 3. The description does not add meaning beyond the schema—it merely references 'custom headers, JSON body, retry, and configurable timeout' which already exist in the schema. No additional parameter semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Make HTTP requests to any REST API' with a clear verb and resource, and lists key capabilities (custom headers, JSON body, retry, timeout) that differentiate it from sibling tools like database-query and file_read. It leaves no ambiguity about the tool's primary function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for making HTTP requests to external REST APIs, but does not explicitly state when to choose this over sibling tools like webhook_send or database-query. There are no exclusion criteria or alternative recommendations, so it only provides an implicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_listA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a meaningful behavioral trait: output is sorted with directories first. However, with no annotations provided, the description carries the full burden and does not specify the exact return format (e.g., names vs. full paths, hidden files, or error handling).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It front-loads the action and adds only the key behavioral detail about sorting, earning every word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one parameter and no output schema, the description covers the primary purpose and the notable sorting behavior. It could be slightly more complete by detailing the return structure (e.g., names only), but it is sufficient for straightforward listing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the only parameter 'path', including its default value. The description adds no additional parameter semantics beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List entries in a directory.' It uses a specific verb and resource, and distinguishes itself from sibling tools like file_read or file_search by focusing on directory enumeration rather than file content or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through the description (use to list directory contents), but there is no explicit guidance on when to prefer this over siblings like file_info or file_search. No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_writeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: creating parent directories and the overwrite flag semantics. However, it does not state what happens when overwrite is false and the file already exists, nor does it mention return values or potential errors. This is partial disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each providing necessary information: the core action and the two behavioral modifiers (parent dir creation, overwrite). No redundant wording or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the main functionality, it lacks information on return values and error behavior when overwrite=false and the file exists. Since no output schema exists, the description should have explained these aspects to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description doesn't need to repeat parameter details. It mentions 'content' implicitly and 'overwrite' explicitly, but adds no syntax or format details beyond the schema. Baseline 3 applies because schema handles the details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Write content to a text file' with a specific verb and resource, clearly distinguishing it from sibling tools like file_read, file_list, file_info, and file_search. It also mentions creating parent directories and overwrite behavior, further clarifying its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: write text content, create parent directories as needed, and set overwrite to true to replace existing files. It does not explicitly mention alternatives or when-not-to-use conditions, but the use case is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• webhook_sendA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context about idempotency keys and HMAC-SHA256 signatures, but it does not disclose the return value, error behavior, or side-effect nature beyond 'send', nor does it mention requirements like authentication or rate limits. This is a moderate level of transparency, not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action and scope, followed by two supporting capabilities. Every word earns its place, with no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description should explain what the tool returns or what happens after invocation. It does neither. It also does not mention behavior when no webhooks match the event type or the possibility of asynchronous delivery. While the tool is relatively simple, this missing return/error context makes it partially incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description repeats the idempotency key purpose already in the schema and adds mention of HMAC-SHA256 signatures, but this does not add meaning to the individual parameters beyond what the schema already provides. The added signing context is more about behavior than parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Send') and clearly states the resource ('all registered webhooks matching the event type'). It effectively distinguishes itself from the sibling tool webhook_register, which is about registration rather than sending.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool when you need to trigger a notification to webhooks based on an event type. It implies the alternative of registering webhooks via webhook_register but does not explicitly name it or provide exclusions. Still, the guidance is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of transparency. It clarifies that it works for both files and directories and lists the types of metadata returned. However, it does not disclose potential error behavior (e.g., nonexistent path) or whether it returns anything beyond those fields. This is adequate but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every word contributes meaning, listing the verb, resource, and key metadata fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one required parameter and no output schema. The description provides sufficient context for what the tool does and what metadata it returns. It lacks explicit return-structure details, but this is acceptable given the low complexity and the clear list of metadata fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter ('path') with a description that largely mirrors the tool description ('Relative path to the file or directory'). Since schema coverage is 100%, the description adds little beyond the schema. It confirms the path can target a file or directory, but this is already in the schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and resource ('metadata about a file or directory'), and explicitly lists the metadata fields (size, modification date, permissions). This clearly distinguishes it from sibling tools like file_read (content), file_write (writing), and file_list (listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: when you need metadata rather than content or operations. It does not explicitly mention alternatives or exclusions, but the purpose is clear enough for a simple read-only tool. No explicit when-not guidance, but the scope is self-evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_clearA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly warns 'This action is irreversible,' disclosing a critical destructive trait. Since no annotations are provided, this disclosure carries the full behavioral burden, and while it doesn't detail all potential side effects, it covers the most important safety aspect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short sentences: the action and a warning. Every word earns its place, with no redundant details or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description conveys the core operation and its irreversibility clearly. It doesn't describe return values or error handling, but for a destructive clear operation, this level of detail is generally sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a complete description for the sole parameter 'collection' (100% coverage). The tool description adds no parameter-specific meaning beyond what the schema states, so it provides no bonus over the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Clear'), the object ('all indexed documents'), and the scope ('from a collection'), making it distinct from sibling tools like semantic_index and semantic_search. The purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to permanently remove all indexed documents from a collection, but it does not explicitly state when to use it versus alternatives, nor does it mention any prerequisites or exclusions. Usage context is implied rather than explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_searchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that results are top-k ranked by cosine similarity and that indexing is required beforehand. However, it does not mention what happens if the collection is missing or not indexed, nor describe the output schema or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using only two sentences to convey purpose, result type, ranking method, and a crucial prerequisite. Information is front-loaded and every clause adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple search tool with complete schema coverage, the description covers the essential purpose, the ranking method, and the indexing prerequisite. It does not detail the return structure or error handling, but the absence of an output schema and the straightforward nature of the task make this acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes each parameter, so the baseline is 3. The description adds general context about top-k and cosine similarity but does not add new information about parameter syntax or specific semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: searching a collection for semantically similar documents to a query. It also specifies the ranking mechanism (cosine similarity) and differentiates from siblings by emphasizing the semantic nature and prerequisite indexing, distinguishing it from database-query or file_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: use when you need semantic similarity search on an indexed collection. It explicitly states the precondition that the collection must be indexed first with semantic_index, which is a valuable cue against alternatives. However, it does not explicitly mention when not to use it or contrast with keyword-based search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• database-queryA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key safety trait: mutations are allowed only if 'allow_mutation: true' is set, and it explicitly lists SQL statement types (SELECT, INSERT, UPDATE, DELETE). While it does not mention return formats or side effects, the mutation guard and operation scope provide meaningful behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place. The first sentence states the core function, the second provides a discovery workflow step, and the third adds a safety condition. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters and no output schema, but the description covers the essential operational aspects: allowed SQL types, the 'list-tables' operation, and the mutation guard. It does not explain the return value format, but for a SQL query tool this is often implicit. The description is complete enough for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by clarifying when to use certain parameters: the 'list-tables' operation for discovery and the 'allow_mutation' requirement for mutating statements. This goes slightly beyond the schema's own descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: "Execute SQL queries against a SQLite database (SELECT, INSERT, UPDATE, DELETE)." It immediately distinguishes this from sibling tools (e.g., api-connector, file_read) by specifying the database context. It also mentions the two operation modes, 'query' and 'list-tables', giving a precise scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool: "Use operation 'list-tables' to discover available tables" and "Mutation queries require 'allow_mutation: true' for safety." It does not explicitly name alternatives or exclusions, but the SQL-focused purpose implies when it is appropriate, earning a 4 rather than 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_indexA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden and does well by disclosing automatic chunking, local ML embedding, no API calls, and document_id-based replacement. It omits some edge-case behavior such as what happens if a collection doesn't exist or expected return values, but the core traits are clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, with the primary action front-loaded and the replacement behavior placed second. No filler or repetition of the input schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The input schema fully documents parameters and the description covers purpose, local processing, and replacement. However, with no output schema and no annotations, it leaves a few gaps like return value, synchronization behavior, and error conditions, preventing a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful parameter semantics by explaining that text is automatically chunked/embedded and by clarifying that providing document_id replaces an existing document, going beyond the schema's bare field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Index a document into a collection for semantic search', using a specific verb ('Index'), a clear direct object ('document'), and a destination ('collection'). This fully distinguishes the tool from siblings like semantic_search and semantic_clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly states the use case (semantic search indexing) and adds context that text will be processed locally with no API calls, which helps an agent decide if this is appropriate. It does not explicitly exclude alternatives or name sibling tools, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• task_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It goes beyond basic get/update by explicitly stating supported transitions ('pending->running, running->completed/failed') and the side effect of firing a webhook when reaching a terminal state. This is valuable context about mutating behavior and timing of side effects, though it doesn't cover permissions or reversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, both front-loaded with the core purpose. The first sentence states the action, and the second adds behavioral constraints. No fluff or redundancy; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters and no output schema, the description covers the key aspects: purpose, transitions, and webhook side effect. It doesn't explicitly describe the return shape, but the get/update nature implies a return of status. Given the moderate complexity and existing schema coverage, it is reasonably complete, though it could mention pagination or response details for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by explaining the allowed transition sequence (pending->running, running->completed/failed) and linking the 'result' parameter to terminal states. This clarifies the semantics of 'update_status' and 'result' better than the raw schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's dual purpose: 'Get the current status of an async task, or update its status.' This is a specific verb+resource pairing that distinguishes it from sibling tools like task_create, which creates tasks. The name 'task_status' aligns perfectly with the described functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage in the context of async task management by defining status transitions and the webhook side effect. While it doesn't explicitly name alternatives, the sibling list includes task_create, making it clear this tool is for checking/updating existing tasks. The context is sufficiently clear, though it lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.